[FEA] Improve GpuJsonToStructs performance

[ttnghia]

The performance of our current GpuJsonToStructs is not good. When running the profiling, it looks like this:

In the particular test case for the profiling above, the only useful work is only what to the end of the read_json range (just above 300ms), which is less than 50% of the entire GpuJsonToStructs projection (>800ms). The rest are just overhead, but it consists mostly of hundreds of small kernel calls and stream syncs due to pure copying data from the intermediate result to the final output.

We can do a lot better by reducing the unnecessary overhead, or improving them by a way that they can run in a much less time. If we divide the runtime of GpuJsonToStructs into sections:

The improvement can be done by the following tasks:

• Section 1: Improve libcudf's cudf::read_json. This needs help from cudf team. Beyond that, we can improve the performance of this section with some auxiliary work:
  • Implement concat_json to join JSON strings given by strings column spark-rapids-jni#2457
  • Adopt JSONUtils.concatenateJsonStrings for concatenating JSON strings #11549
• Section 2: Improve the process that assembles the output table from cudf::read_json into the output structs column with the desired read schema. Currently, this process may need to copy a lot of columns from the output table of cudf::read_json (hundreds columns), which is a significant overhead. We can see it from the profiling of this section. We can just move them instead.
  • cudf issue: [FEA] Implement a better JNI function to assemble the output columns from cudf::read_json rapidsai/cudf#17002
• Section 3: Improve the conversion step from strings columns into the desired types. Some columns need to be converted but some are just output directly without any conversion. However, instead of being moved into the output, they are again copied and that causes a lot of overhead if the number of strings columns is significant.
  • spark-rapids-jni issue: [FEA] Implement a function to convert a list of strings columns into a structs column with desired schema spark-rapids-jni#2468