Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when using %sqlplot in snowflake #697

Merged
merged 44 commits into from
Aug 18, 2023

Conversation

tonykploomber
Copy link

@tonykploomber tonykploomber commented Jun 30, 2023

Describe your changes

Fix %sqlplot histogram issue on snowflake

Original Problem - Table name and Column are uppercase

Actually our original internal histogram SQL query can perform the function

In the test_sqlplot_histogram, we use plot_something data to test. During the initial data loading stage, the table name and column are UPPERCASE

We will need to test against the uppercase table name and column name in the snowflake case

Screenshot 2023-07-21 at 2 54 09 PM

Solution

Make test_sqlplot_histogram test against PLOT_SOMETHING_HASHKEY instead of plot_something_hashkey table since it's created as uppercase

Issue number

Closes #444

Checklist before requesting a review


📚 Documentation preview 📚: https://jupysql--697.org.readthedocs.build/en/697/

@tonykploomber tonykploomber linked an issue Jun 30, 2023 that may be closed by this pull request
@tonykploomber tonykploomber marked this pull request as draft June 30, 2023 01:01
@tonykploomber tonykploomber added allow-workflow-edits Allow edits to .github/workflows allow-broken-links labels Jul 19, 2023
@tonykploomber tonykploomber marked this pull request as ready for review July 21, 2023 19:20
@tonykploomber
Copy link
Author

@edublancas
Please review this

.github/workflows/ci-integration-db.yaml Outdated Show resolved Hide resolved
.github/workflows/ci.yaml Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
src/sql/connection.py Outdated Show resolved Hide resolved
src/sql/plot.py Outdated Show resolved Hide resolved
src/sql/plot.py Outdated Show resolved Hide resolved
src/sql/plot.py Outdated Show resolved Hide resolved
src/sql/plot.py Outdated Show resolved Hide resolved
src/tests/integration/test_generic_db_operations.py Outdated Show resolved Hide resolved
src/tests/integration/test_questDB.py Outdated Show resolved Hide resolved
Copy link

@yafimvo yafimvo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tonykploomber I'm not sure I understood the solution.

Let's take the following table from our Snowflake connection

image

Since running %sqlplot histogram --table plot_something_e1bde5 -c y will raise the original error, I ran the following command:

image

It looks like our histogram is incorrect. It should be something like this

image

I went over the skipped tests artifacts and they look the same,

e.g.

expected:
histogram_stacked_large_bins-expected

results:
histogram_stacked_large_bins

src/sql/plot.py Outdated Show resolved Hide resolved
src/sql/plot.py Show resolved Hide resolved
src/sql/util.py Outdated
def to_upper_if_snowflake_conn(conn, upper):
return (
upper.upper()
if isinstance(conn, AbstractConnection)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to check if this is an abstract connection? do anything fails if we don't?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanna make sure there is callable function we can invoke
fix in 4a4f218

@edublancas
Copy link

@tonykploomber this PR hasn't received any updates in two weeks. please address the review so we can merge it

@tonykploomber tonykploomber removed the no-changelog This PR doesn't require a changelog entry label Aug 15, 2023
@tonykploomber
Copy link
Author

@edublancas Apologize for missing updates

Please check again

Copy link

@edublancas edublancas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also update the snowflake tutorial showing the %sqlplot

@@ -11,6 +12,7 @@

__all__ = [
"ConnectionManager",
"AbstractConnection",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this import? I don't see it used anywhere

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in 7ee92a9

@edublancas
Copy link

@tonykploomber fix the merge conflicts as well

@edublancas
Copy link

@tonykploomber is this ready for review? if so, don't forget to request a review, otherwise I don't know if I review or wait

@tonykploomber
Copy link
Author

@edublancas
Just made the plotting section in docs, added in 9cf70da

Preview: https://jupysql--697.org.readthedocs.build/en/697/integrations/snowflake.html#plotting

@edublancas edublancas merged commit 8bcf3ad into master Aug 18, 2023
22 checks passed
@edublancas edublancas deleted the 444-error-when-using-sqlplot-in-snowflake branch August 18, 2023 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error when using %sqlplot in snowflake
3 participants