How to create a list of static_maps that support lookups on GPU.

[Gibyeng]

Is it possible to create100000+ static_maps with different settings (e.g. different sizes) that support lookups on GPU?
I make a statement like this "std::vector<cuco::static_map<int,int>> maps(NUM_MAPS);", but got stuck when I try to assign each map properly. Hope anyone could help me. Thanks in advance.

[Gibyeng]

Hi, I just find an related answer here, which just solves my problem.
BTW, I plan to search the key-value pairs stored in each of those maps, and get the top-k pairs of those maps (just like a priority queue). Of course, this can be done by a kernel function (insert then reduce). I am wondering whether this can be achieved more efficiently. Could you please offer me some advices?

[sleeepyjack]

Hi @Gibyeng,

We have a for_each function (both host and device API) which iterates over each filled slot in the map and applies a user-provided functor for each key/value pair found. If I understand your problem correctly, you could define a functor that performs the top-k reduction. Would that work for you?

[Gibyeng]

Cool, I think the for_each API is the one I am looking for. Thanks for your advises.

[Gibyeng]

Unfortunately, after I test the code in my own machine. My compiler reports a cuda error (cudaErrorInvalidValue) at line 36 (the std::move method).
I am using nvcc 11.8 & g++-11 for compiling.
Are there any other alternatives I can do rather than the "std::move" methods?

[PointKernel]

https://godbolt.org/z/3ss7cTPjj

How do you add cuco into your code? It seems that you not fetching the proper CCCL version per https://github.com/NVIDIA/cuCollections?tab=readme-ov-file#building-cucollections

[Gibyeng]

Hi, @PointKernel
I download CCCL and cuco from github, then I link the header files to my project by Cmake by target_include_directories.

#Link cuco
target_include_directories(CuCollectionsExample PRIVATE /home/xxx/Documents/cuCollections/include)
#Link CCCL
target_include_directories(CuCollectionsExample PRIVATE /home/xxx/Documents/cccl/thrust /home/xxx/Documents/cccl/libcudacxx/include /home/xxx/Documents/cccl/cub)

[Gibyeng]

I try to build the examples in the cuco repo. There are also errors when building STATIC_MAP_TEST. One of them looks like this:

/home/xxx/Documents/cuCollections/build/_deps/cccl-src/libcudacxx/lib/cmake/libcudacxx/../../../include/cuda/functional:190:26: error: could not convert ‘cuda::std::__4::__invoke<__nv_dl_wrapper_t<__nv_dl_tag<void (*)(), CATCH2_INTERNAL_TEMPLATE_TEST_0<key_pair_type<long int>, value_pair_type<int> >, 3> >, int>((* & cuda::std::__4::move<__nv_dl_wrapper_t<__nv_dl_tag<void (*)(), CATCH2_INTERNAL_TEMPLATE_TEST_0<key_pair_type<long int>, value_pair_type<int> >, 3> >&>(((cuda::__4::__detail::__return_type_wrapper<cuco::pair<key_pair_type<long int>, value_pair_type<int> >, __nv_dl_wrapper_t<__nv_dl_tag<void (*)(), CATCH2_INTERNAL_TEMPLATE_TEST_0<key_pair_type<long int>, value_pair_type<int> >, 3> > >*)this)->cuda::__4::__detail::__return_type_wrapper<cuco::pair<key_pair_type<long int>, value_pair_type<int> >, __nv_dl_wrapper_t<__nv_dl_tag<void (*)(), CATCH2_INTERNAL_TEMPLATE_TEST_0<key_pair_type<long int>, value_pair_type<int> >, 3> > >::__fn_)), (* & cuda::std::__4::forward<int>((* & __as#0))))’ from ‘int’ to ‘cuco::pair<key_pair_type<long int>, value_pair_type<int> >’

[PointKernel]

We build examples and benchmarks in our CI pipeline, and we haven't encountered the error you mentioned. Could you try adding cuco to your project using CPM (https://github.com/NVIDIA/cuCollections?tab=readme-ov-file#adding-cucollections-to-a-cmake-project)? I suspect there might be a version mismatch between CCCL in your project and the one required by cuco.

[Gibyeng]

Hi @PointKernel ,
I think I know the reason, the code you provided here also produces the exact same error after execution.

terminate called after throwing an instance of 'cuco::cuda_error'
  what():  CUDA error at: /opt/compiler-explorer/libs/cucollections/dev/include/cuco/utility/allocator.hpp60: cudaErrorInvalidValue invalid argument
Program terminated with signal: SIGSEGV

[Gibyeng]

@sleeepyjack @PointKernel Hi, sorry to interrupt you again. Any idea about how to slove the memory deallocation problem in the example when execute the code. It seems a small problem but real confuses me. Thank you in advance.

[PointKernel]

https://godbolt.org/z/8M35av8Wh

There's an issue with the move constructor, which should have been disabled. Using unique pointers for hash tables could resolve this.

Once fixed, you might notice that the code is time-consuming due to expensive hash table construction, which involves device memory allocation. Instead of creating separate hash tables, probably worth considering using bulk storage as shown in this example. This can significantly optimize your performance.

[PointKernel]

Opened #612 to track the bug

[Gibyeng]

@PointKernel Thanks a lot, I will try both methods. BTW, This work is great, it saves me a lot of efforts to implement an efficient GPU hash table myself.