On 10/21/2011 08:38, Miroslav Lachman wrote:
> Hi, I am back on this topic...
> Ivan Voras wrote:
>> On 14/10/2011 11:20, Miroslav Lachman wrote:
>>> Hi all,
>>> I tried some tuning of dirhash on our servers and after googlig a bit, I
>>> found an old GSoC project wiki page about Dynamic Memory Allocation for
>>> Dirhash: http://wiki.freebsd.org/DirhashDynamicMemory
>>> Is there any reason not to use it / not commit it to HEAD?
>> AFAIK it's sort-of already present. In 8-stable and recent kernels you
>> can give huge amounts of memory to dirhash via vfs.ufs.dirhash_maxmem
>> (but except in really large edge cases I don't think you *need* more
>> than 32 MB), and the kernel will scale-down or free the memory if not
>> In effect, vfs.ufs.dirhash_maxmem is the upper limit - the kernel will
>> use less and will free the allocated memory in low memory situations
>> (which I've tried and it works).
> So the current behavior is that on 7.3+ and 8.x we have smaller average
> dirhash buffer (by default) than it was initialy 10 years ago. Because
> it starts as 2MB fixed size and now we have 2MB max, which is lowered in
> low mem situations... and sometimes it is set to 0MB!
> I caught this 2 days ago:
> root@rip ~/# sysctl vfs.ufs
> vfs.ufs.dirhash_reclaimage: 5
> vfs.ufs.dirhash_lowmemcount: 36953
> vfs.ufs.dirhash_docheck: 0
> vfs.ufs.dirhash_mem: 0
> vfs.ufs.dirhash_maxmem: 8388608
> vfs.ufs.dirhash_minsize: 2560
> I set maxmem to 8MB in sysctl.conf to increase performance and
> dirhash_mem 0 is really bad surprise!
> I am worrying about bad performance in situation where dirhash is
> emptied in situations, where server is already running at maximum
> performance (there is some memory hungry process and system can start
> swapping to disk + dirhash is efectively disabled)
> I found a PR kern/145246
> Is it possible to add some dirhash_minmem limit to not clear all the
> dirhash memory?
> So I can set dirhash_minmem=2MB dirhash_maxmem=16MB and then dirhash_mem
> will be allways between these two limits?
Isn't that what vfs.ufs.dirhash_minsize is for? I think given that there
is a lot more memory in modern systems setting that higher by default is
probably a good idea. Or maybe I'm misunderstanding what that knob does?
>>> And second question - is there any negative impact with higher
>>> vfs.ufs.dirhash_maxmem? It stil defaults to 2MB (on FreeBSD 8.2) after
>> Not that I know of.
>>> 10 years, but I think we all are using bigger FS in these days with lot
>>> of files and directories and 2MB is not enough.
>> AFAIK I've changed it to autotune so it's configured to approximately 4
>> MB on a 4 GB machine (and scales up) in 9.
> I didn't tried 9 yet. Does it mean dirhash_maxmem is initially set to
> approximately 1% of physical RAM and then it can be set higher by sysctl
> as in older versions?
I'm not sure that's what's happening, I have 6G of ram in this box and I
have this by default:
Nothin' ever doesn't change, but nothin' changes much.
-- OK Go
Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price. :) http://SupersetSolutions.com/