Loading ...
Sorry, an error occurred while loading the content.

5184Re: When we cannot waste 1MB per datum.

Expand Messages
  • Omer Zak
    Jun 25, 2012
    • 0 Attachment
      Nice to remind the world that I existed also at 2002. :-)
      Obviously, representing a single FC position is not something that
      should, need to or even could consume a full 1MB RAM.

      What I had in mind was AI stuff.

      --- Omer

      On Mon, 2012-06-25 at 17:14 +0300, Shlomi Fish wrote:
      > Hi all,
      > Omer Zak (CCed to this message) wrote this old (2002) Hackers-IL message:
      > http://tech.groups.yahoo.com/group/hackers-il/message/1894
      > Titled "How to exploit 1MB data structures?" this message reads:
      > <<<<
      > The future computers will have terabytes of memory (mostly in disks, but
      > some in RAM). They will be able to manipulate megabytes at the time
      > needed by yesterday's computers to manipulate single bytes.
      > I wonder how can all those extra bytes be harnassed. (AI anyone?)
      > Let's try to use 1MB to represent the string "The beep has booped".
      > The data structure will contain:
      > >>>>
      > Now I'd like to give a view from the other side of the fence: when have
      > milliards (10^9s) of data items and we need to fit them all in RAM (first -
      > 128 GB - now I also got access to a machine with 512 GB), because storing and
      > retrieving them on disk is far too slow. This is the case of Freecell Solver's
      > dbm_fc_solver where I've spent a lot of time reducing its RAM consumption:
      > * http://tech.groups.yahoo.com/group/fc-solve-discuss/message/1114
      > And there's another optimisation coming up:
      > * http://tech.groups.yahoo.com/group/fc-solve-discuss/message/1115
      > If every position in the game’s graph consumed 1MB of RAM, then I could fit
      > at most 512*1024 of them in 512 GB of RAM, which is all I have at the moment.
      > And I can reach much more than 512K of positions on my underpowered home
      > computers. Furthermore, manipulating these 1MB data will likely be much more
      > time-consuming than the compact ones I'm using at the moment. Some of my scans
      > reached 3 milliard stored positions, and such an amount would have consumed
      > roughly 3 peta bytes (3 * 10^15), which I suspect are beyond any single HPC
      > machine at the moment (to say nothing of consumer computers).
      > I think that if we have large amounts of RAM resources at our disposal, we
      > should not carelessly waste them like Omer suggests (unless of course we ran
      > into a use-case that necessitates that).
      > Regards,
      > Shlomi Fish

      PHP - the language of the Vogons.
      My own blog is at http://www.zak.co.il/tddpirate/

      My opinions, as expressed in this E-mail message, are mine alone.
      They do not represent the official policy of any organization with which
      I may be affiliated in any way.
      WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html