Enhance current integration between Kedro and Ibis

[deepyaman]

Description

Given the existing (admittedly-basic) Ibis integration (via ibis.TableDataset) has garnered interest and usage, we would like to take some time to address some of the enhancements requested by users (and others that will generally improve Kedro user experience with Ibis backends).

The priorities below are an initial rough prioritization; please comment if certain things (items below and those not yet on the list) would be important for you, your team, or to users you've heard from!

Issues

• [P0] Add write support for ibis.TableDataset to files kedro-plugins#828
• [P0] Support writing files to PyArrow/fsspec-compatible filesystems
• [P0] feat(datasets): support setting Ibis table schemas kedro-plugins#833
• [P0] Support for inserting data in ibis.TableDataset  kedro-plugins#834
• [P0] Support for Upsert functionality in ibis.TableDataset kedro-plugins#835
• [P1] Enhance documentation with tutorials and working examples of how to run SQL queries with Ibis #4189
• [P1] Add Kedro-Viz table preview for Ibis datasets
• [P1] Support passing credentials to Ibis datasets (without custom resolver)
• [P2] Misleading Table not found error when using wrong database kedro-plugins#725
• [P2] Support validation of Ibis tables via the Kedro-Pandera integration ([QUESTION] Could integration with Ibis be supported? Galileo-Galilei/kedro-pandera#88, pending work in Support Ibis Backend unionai-oss/pandera#1105)

[deepyaman]

Tagging some people I've seen active on both Kedro and Ibis repos or that I know have played with the integration, as would love your thoughts on whether I'm missing things that you all have run into and would like to see addressed, as well as your views on respective priorities of these issues!

@mark-druffel @inigohidalgo @datajoely @galenseilis @gitgud5000 @filipeo2-mck @astrojuanlu @vishu1994 @RicardoQuirozQB @gtauzin

[galenseilis]

Related: Galileo-Galilei/kedro-pandera#88 (comment)

[deepyaman]

Related: Galileo-Galilei/kedro-pandera#88 (comment)

Thanks @galenseilis! I've gone ahead and linked that above; didn't know there was already an issue you'd opened covering it. 😄