Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🍀 Reduce unnecessary requests by disabling sagemaker_check() explicitly in Blocks #9543

Closed
1 task done
vmatt opened this issue Oct 5, 2024 · 1 comment · Fixed by #9546
Closed
1 task done

Comments

@vmatt
Copy link
Contributor

vmatt commented Oct 5, 2024

  • I have searched to see if a similar issue already exists.

Is your feature request related to a problem? Please describe.

I'm developing a gradio app, which will be used in a environment without internet access.
I noticed, that To create a public link, set 'share=True' in 'launch()'. appears very slowly (8-15 seconds) when running on the isolated instance, but it is way faster on my internet-connected development machine.

After turned on .DEBUG log level, i saw that in Blocks.py, there is line: is_sagemaker = utils.sagemaker_check(), which function imports boto3 & invokes a caller identity check against AWS services. This includes sending mutliple requests against AWS.

One thing I quickly realised, that this happens after the local app server starts, so there is no real performance/startup benefit, I can access the Gradio interface in 3-5 seconds no matter what happens later. However, I don't see the point sending requests to AWS each startup by default. I don't know the numbers but I'm guessing there are a substantial amount of the users who are not using Gradio via Sagemaker.

Describe the solution you'd like

So in the name of environmental friendliness 🍀 to reduce unnecessary requests to AWS, I'm proposing two options:

  1. Breaking change: set is_sagemaker default to False, and and create a check_sagemaker flag for Blocks(), which needs to explicitly set to True, if the user is using Sagemaker and want to run utils.sagemaker_check()
  2. Non-breaking change: allow user to define is_sagemaker value explicitly when creating a block:
    with gr.Blocks(analytics_enabled=False, is_sagemaker=False) as interface:

Option 1. as-is for sure a no-go, as any current Sagemaker deployment would need to add check_sagemaker() flag.

So I'll try to implement option 2, and open a PR for a it. But let me know if you have a better idea!

Additional context

Running this on on my local machine, this causes not much issue, but on the isolated environment, boto3 calls times out.

urllib3.exceptions.NameResolutionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x142239250>: Failed to resolve 'sts.amazonaws.com' ([Errno 8] nodename nor servname provided, or not known)

Without internet access

is_sagemaker=None

2024-10-05 13:46:41,604 - INFO - Start
Running on local URL:  http://127.0.0.1:7860
2024-10-05 13:46:45,186 - INFO - HTTP Request: GET http://127.0.0.1:7860/startup-events "HTTP/1.1 200 OK"
2024-10-05 13:46:45,312 - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2024-10-05 13:46:50,773 - INFO - HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
To create a public link, set `share=True` in `launch()`.
  • Time to URL accessible (45.18-41.6): 3.58 sec
  • Time to async requests completed (50.7-41.6): 9.1 sec

is_sagemaker=False

2024-10-05 13:48:26,701 - INFO - Start
Running on local URL:  http://127.0.0.1:7860
2024-10-05 13:48:30,290 - INFO - HTTP Request: GET http://127.0.0.1:7860/startup-events "HTTP/1.1 200 OK"
2024-10-05 13:48:30,316 - INFO - HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
To create a public link, set `share=True` in `launch()`.
  • Time to URL accessible (30.29-26.7): 3.59 sec
  • Time to async requests completed(30.31-26.70): 3.61 sec

With internet access

is_sagemaker=None

2024-10-05 13:53:00,889 - INFO - Start
Running on local URL:  http://127.0.0.1:7860
2024-10-05 13:53:05,111 - INFO - HTTP Request: GET http://127.0.0.1:7860/startup-events "HTTP/1.1 200 OK"
2024-10-05 13:53:05,322 - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2024-10-05 13:53:06,025 - INFO - HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
To create a public link, set `share=True` in `launch()`.
  • Time to URL accessible (5.11-0.89): 4.22 sec
  • Time to async requests completed (6.02-0.89): 5.13 sec

is_sagemaker=False

2024-10-05 17:11:20,441 - INFO - Start
Running on local URL:  http://127.0.0.1:7860
2024-10-05 17:11:24,176 - INFO - HTTP Request: GET http://127.0.0.1:7860/startup-events "HTTP/1.1 200 OK"
2024-10-05 17:11:24,201 - INFO - HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
To create a public link, set `share=True` in `launch()`.
  • Time to URL accessible (24.17-20.44): 3.73 sec
  • Time to async requests completed (24.20-20.44): 3.76 sec
@abidlabs
Copy link
Member

abidlabs commented Oct 5, 2024

Closed via #9546, thanks @vmatt for bringing this to our attention

@abidlabs abidlabs closed this as completed Oct 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants