Re: Please evaluate my understanding wrt access files
- On 10/31/2009 7:39 PM, Stan Hoeppner wrote:
> Noel Jones put forth on 10/31/2009 1:12 AM:"overhead" is pretty much independent of file size. Each
>> Each lookup table requires overhead. 30 separate tables requires
>> considerably more overhead than one table. The size of the dataset
>> doesn't change, it's the overhead that gets smaller. The more
>> concurrent smtpd processes running, the more it matters.
> I guess I was kinda looking for the technical definition of 'overhead'
> here. If the overhead per file is a straight percentage, then it
> shouldn't matter if it's one big file or 30 smaller files.
extra lookup table is another open file, buffer space, another
library call (that might allocate its own buffer), and an
additional lookup. In a tiny environment it's not enough to
make a difference -- probably not enough to even notice.
But one lookup always beats two lookups, one open file always
uses less resources than two open files. Run enough smtpd
processes and it starts to make a difference; at some point it
can make a critical difference. And it's always "prettier"
to do one thing one time vs. doing the same thing over and over.
> What is theMeasure it on your system. Probably not a lot, but it adds up.
> actual memory footprint overhead that's being added by Posfix for each
> map file?
> And is the amount of overhead the same for all map types, orYes, different table types use different system libraries and
> different/variable depending on map type?
have different resource requirements. But for each table type
it's a fairly constant value unaffected by table size (very
small tables might use a little less RAM under some conditions
-- but this really doesn't matter).
> Is it something like 3 bytesNo, it's fixed.
> of overhead added for every 10 bytes in a map file, for example?
> I canFewer files = less overhead. Size doesn't matter here.
> understand that having everything in one map file, or few map files,
> compared to many, can increase lookup performance due to indexing
> efficiency. What I don't understand is how having fewer larger map
> files decreases memory footprint.
> Also, given that cidr files aren't postmap'd, is it safe to assume theyOne lookup beats two every time.
> aren't indexed? If they aren't indexed, how does putting the contents
> in a single file speed up searching?
> If they are indexed when PostfixHow is postfix supposed to know you want all these tables
> starts or reloads, then why wouldn't Postfix created a single index
> _after_ reading in all the data from all the cidr files, creating one
> internal program table?
treated as one? Anyway, postfix would still need to open the
files etc. so all you would save is the extra lookups. It's
easy enough for you to control by building the tables the way
you want them used that postfix shouldn't be guessing.
-- Noel Jones