Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log file messages regrading downloads are misleading #4561

Open
bryank-cs opened this issue Jun 12, 2024 · 0 comments
Open

Log file messages regrading downloads are misleading #4561

bryank-cs opened this issue Jun 12, 2024 · 0 comments

Comments

@bryank-cs
Copy link

The following log messages say that the file being downloaded failed and won't be retried, but then downloads it from another source:

== 2024-06-12 08:26:39,560 filetools.py:832 WARNING URL https://files.pythonhosted.org/packages/cc/1f/676fc9e29c68e9c39c6dadf150ab4e5bf4907de4b9afd2bc6e0afd24ab7c/easybuild_easyblocks-4.9.2.tar.gz was not found (HTTP response code 404), not trying again
== 2024-06-12 08:26:39,562 filetools.py:859 WARNING Download of https://files.pythonhosted.org/packages/cc/1f/676fc9e29c68e9c39c6dadf150ab4e5bf4907de4b9afd2bc6e0afd24ab7c/easybuild_easyblocks-4.9.2.tar.gz to /opt/sources/e/EasyBuild/easybuild_easyblocks-4.9.2.tar.gz failed, done trying
== 2024-06-12 08:26:39,647 filetools.py:816 INFO File size for https://files.pythonhosted.org/packages/5d/85/e8593ceeb00c61253204e74d2a8360076ce016f42d83d33841f8e7de57a1/easybuild_easyblocks-4.9.2.tar.gz: 598689
== 2024-06-12 08:26:39,651 filetools.py:270 INFO Not showing progress bar for downloading small file (size 598689)
== 2024-06-12 08:26:39,924 filetools.py:823 INFO Downloaded file easybuild_easyblocks-4.9.2.tar.gz from url https://files.pythonhosted.org/packages/5d/85/e8593ceeb00c61253204e74d2a8360076ce016f42d83d33841f8e7de57a1/easybuild_easyblocks-4.9.2.tar.gz to /opt/sources/e/EasyBuild/easybuild_easyblocks-4.9.2.tar.gz
== 2024-06-12 08:26:39,925 filetools.py:855 INFO Successful download of file easybuild_easyblocks-4.9.2.tar.gz from url https://files.pythonhosted.org/packages/5d/85/e8593ceeb00c61253204e74d2a8360076ce016f42d83d33841f8e7de57a1/easybuild_easyblocks-4.9.2.tar.gz to path /opt/sources/e/EasyBuild/easybuild_easyblocks-4.9.2.tar.gz
== 2024-06-12 08:26:39,925 easyblock.py:951 INFO Successfully downloaded source file easybuild_easyblocks-4.9.2.tar.gz from https://files.pythonhosted.org/packages/5d/85/e8593ceeb00c61253204e74d2a8360076ce016f42d83d33841f8e7de57a1/easybuild_easyblocks-4.9.2.tar.gz

This is from the EasyBuild-4.9.2.eb that is automatically generated using --install-latest with:

source_urls = [
    # %(namelower)s-framework
    'https://files.pythonhosted.org/packages/cc/1f/676fc9e29c68e9c39c6dadf150ab4e5bf4907de4b9afd2bc6e0afd24ab7c/',
    # %(namelower)s-easyblocks
    'https://files.pythonhosted.org/packages/5d/85/e8593ceeb00c61253204e74d2a8360076ce016f42d83d33841f8e7de57a1/',
    # %(namelower)s-easyconfigs
    'https://files.pythonhosted.org/packages/99/b2/d899b4310bc54a10e0fb46995a2abc333857db16d116f22a53b0313d13d7/',
]
sources = [
    'easybuild_framework-%(version)s.tar.gz',
    'easybuild_easyblocks-%(version)s.tar.gz',
    'easybuild_easyconfigs-%(version)s.tar.gz',
]

It could be generated using a sources syntax that associates the correct URL for each file.

In other cases with multiple source urls, the log messages should be written to clarify it will retry with a different URL, if possible. Or maybe change the log levels to show successful downloads only, and all the failures only if none worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant