[dpdk-dev] DPDK Hash library

[Matthew Hall]

On Thu, Jul 02, 2015 at 05:55:20PM +0000, De Lara Guarch, Pablo wrote:
> You are probably talking about extendable buckets here.
> The downsize of that approach is that you have to allocate memory on the fly,
> whereas with the cuckoo hash implementation, the entry can be stored in an alternative bucket
> without having to reserve more memory (which also will take you more time).
> With this approach, hash tables can get a higher utilization, as other less used
> buckets can be used to store keys from other busier buckets.
> 
> Pablo

Expanding and shrinking buckets constantly can also be concurrency-hostile, 
and is a lot more complicated to get right than just using a good rehash 
algorithm and a nice static hunk of memory on contiguous hugepages for minimal 
TLB / cache pressure.

If you want to do these more complex manipulations uthash is probably a better 
route. But it will be slower than the DPDK hashes by quite a ways I think. I 
used DPDK hash for my TCP socket table where everything is a very predictable 
size, but I had to use uthash for my unpredictably sized byte buffers for 
security indicators (IP, URL, Domain, Email, File Hash, etc.)

Of course, when you do this kind of stuff in your app it is going to give you 
scaling problems and you'll have to spend a lot of time tuning it.

Matthew.