Optimizing IVF-PQ Performance with RAPIDS cuVS: Key Tuning Techniques

Within the first a part of the sequence, an outline of the IVF-PQ algorithm was offered, explaining its basis on the IVF-Flat algorithm and using Product Quantization (PQ) to compress the index and assist bigger datasets. Partially two, the main focus shifts to the sensible elements of tuning IVF-PQ efficiency, which is essential for attaining optimum outcomes, particularly when coping with billion-scale datasets.

Tuning Parameters for Index Constructing

IVF-PQ shares some parameters with IVF-Flat, resembling coarse-level indexing and search hyper-parameters. Nonetheless, IVF-PQ introduces further parameters that management compression. One of many essential parameters is n_lists, which determines the variety of partitions (inverted lists) into which the enter dataset is clustered. The efficiency is influenced by the variety of lists probed and their sizes. Experiments counsel that n_lists within the vary of 10K to 50K yield good efficiency throughout recall ranges, although this may fluctuate relying on the dataset.

One other essential parameter is pq_dim, which controls compression. Beginning with one fourth the variety of options within the dataset and growing in steps is an efficient approach for tuning this parameter. Determine 2 within the authentic weblog submit illustrates important drops in QPS, which will be attributed to components resembling elevated compute work and shared reminiscence necessities per CUDA block.

The pq_bits parameter, starting from 4 to eight, controls the variety of bits utilized in every particular person PQ code, affecting the codebook dimension and recall. Decreasing pq_bits can enhance search velocity by becoming the look-up desk (LUT) in shared reminiscence, though this comes at the price of recall.

Extra Parameters

The codebook_kind parameter determines how the codebooks for the second-level quantizer are constructed, both for every subspace or for every cluster. The selection between these choices can influence coaching time, GPU shared reminiscence utilization, and recall. Parameters resembling kmeans_n_iters and kmeans_trainset_fraction are additionally necessary, although they hardly ever want adjustment.

Tuning Parameters for Search

The n_probes parameter, mentioned within the earlier weblog submit on IVF-Flat, is crucial for search accuracy and throughput. IVF-PQ supplies further parameters like internal_distance_dtype and lut_dtype, which management the illustration of distance or similarity throughout the search and the datatype used to retailer the LUT, respectively. Adjusting these parameters can considerably influence efficiency, particularly for datasets with massive dimensionality.

Enhancing Recall with Refinement

When tuning parameters isn’t sufficient to realize the specified recall, refinement gives a promising different. This separate operation, carried out after the ANN search, recomputes actual distances for chosen candidates and reranks them. The refinement operation can considerably enhance recall, as demonstrated in Determine 4 of the unique weblog submit, although it requires entry to the supply dataset.

Abstract

The sequence on accelerating vector search with inverted-file indexes covers two cuVS algorithms: IVF-Flat and IVF-PQ. IVF-PQ extends IVF-Flat with PQ compression, enabling quicker searches and the flexibility to deal with billion-scale datasets with restricted GPU reminiscence. By fine-tuning parameters for index constructing and search, knowledge practitioners can obtain the very best outcomes effectively. The RAPIDS cuVS library gives a variety of vector search algorithms to cater to varied use instances, from actual searches to low-accuracy-high-QPS ANN strategies.

For sensible tuning of IVF-PQ parameters, confer with the IVF-PQ pocket book on GitHub. For extra particulars on the supplied APIs, see the cuVS documentation.

Picture supply: Shutterstock