Re: Chunked mempools, a first verdict

On 2 May 2001, at 22:05, Henrik Nordstrom <hno@hem.passagen.se> wrote:

> > > No, it is probably a good name, but some stuff should probably move to
> > > mem.c (maybe even all of it).
> >
> > Are you *sure*? I'd disagree, because stuff in MemPoolStats.c is very
> > specific to current memPool implementation, while mem.c is somewhat
> > free from specifics. Moving lots of stuff to mem.c will clutter it with
> > stuff that someday may be changed alot.
>
> The way to specific stuff should move into lib, with a suitable API that
> is not soo specific.
>
> > I'm in process of hiding most mempool internal funcs behind a Control
> > function. Then I could move most of actual work to lib/ but we'd still
> > need wrappers under src/ for what needs public declarations.
> > I just wonder if that is a right path.
>
> Same thing, different call method. Wait with that change. Having a
> control function only complicates matters. Even if you have a control
> function like that you still have an API of the same number of
> functions, only thing changed is that you have hidden them behind one
> object symbol name. In afct, as proposed the effect only is that you are
> using one symbol more than if not having the control function (in theory
> each macro also counts as a symbol, but is a bit harder to count with
> automatic measures)

 damn. I just finished with that... ;-/

 You guys are confusing hell out of me. What is the API you are after?
 Public API is carved in stone:

memPoolAlloc(MemPool *);
memPoolCreate(char *, size_t);
memPoolFree(MemPool *, void *);
memPoolDestroy(pool);

 All else is bunch of hacks and needed/useful helpers.
 Eg. someone have put stats for memPool allocs/frees on the Info page.
 For that he have added two globals to the zoo:

extern unsigned int mem_pool_alloc_calls;
extern unsigned int mem_pool_free_calls;

 Is that now part of an API?

 Ok, lets go though it again:
memPoolInit() - we need that, don't we?
memPoolConfigure - need to change mem_idle_limit
memPoolInUseCount(pool) - to get number of items in use for a pool
memPoolCleanModule() - destroy all idle pools, at shutdown (and reconfig?)
memPoolsTotalAllocated() - get total allocated memory, for Info page

 Now specific to chunked mempools:
memPoolCleanIdlePools - need periodic cleanup function, with access to event.c
   to reschedule itself. Needs access to worker funcs under lib/ or must be
   implemented in full under lib/
memPoolTune - useful, to tune chunk sizes
memPoolFlushMeters - need to get accounting meters uptodate before reports
memPoolReport() - cachemgr reporter
memMeterSyncHWater() - if we want high water mark accounting.

 Now, either funcs or data structs to get statistics reported:
TheMeter - accounting totals for all pools
Array Pools - to traverse perpool accounting stats for reporting

 What of the above can we discard in principle??

 Stuff with duality due to lib/ vs. src/ split:
memPoolInit() - is run as early as possible. To implement, you either add
  a worker func in lib/ or share Array Pools data struct and implement in src/
  Either way - 2 symbols at least
memPoolConfigure - you either pass a new limit to a lib/ func, or share
  global variable
memPoolCleanIdlePools() - no way you can reschedule event from under lib/
  you either have a public func to implement, or share data structs to
  implement in under src/
memPoolCleanModule() - use of debug() forces it to be wrapped in src/ -
  2 symbols: worker and wrapper
memPoolReport() - needs tons of stuff from lib/, yet has to be done under src/
  as it writes to StoreEntry - 2 symbols at least.
 etc..

 I'm at total loss as to whats wrong with above public symbols.
 To undo the control function is not problem. If that is unwanted,
 I'd revert back to old style. Just please be clear on your preference.
 I just think that at least some functions are well suited for a Control
 function (like Tune, Init, Flush, CleanModule)

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Delfi Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Wed May 02 2001 - 16:04:42 MDT