brew return Error: Too many redirects (max = 5)

[wykeinstein]

I use brew install virt-manager in the zsh shell，after a while，the brew stopped and return "Error: Too many redirects (max = 5)"

How can i solve this problem

[Bo98]

Can you run with HOMEBREW_CURL_VERBOSE=1 and show (copy-paste rather than screenshot) the output?

We'll raise the limit to 25 but it would be interesting to know why I can't reproduce the issue. I only get 4 hops but you are getting at least 6.

[wykeinstein]

I got the output of curl --head --location https://pypi.io/packages/source/i/idna/idna-2.8.tar.gz，is as follows:

jason@osx downloads % curl --head --location https://pypi.io/packages/source/i/idna/idna-2.8.tar.gz
HTTP/1.1 200 Connection established

HTTP/2 301 
server: Varnish
retry-after: 0
location: https://pypi.org/packages/source/i/idna/idna-2.8.tar.gz
content-type: text/html; charset=UTF-8
accept-ranges: bytes
date: Wed, 27 Apr 2022 14:03:21 GMT
x-served-by: cache-bur-kbur8200111-BUR
x-cache: HIT
x-cache-hits: 0
x-timer: S1651068202.501915,VS0,VE0
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-frame-options: deny
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
content-length: 122

HTTP/1.1 200 Connection established

HTTP/2 301 
content-security-policy: base-uri 'self'; block-all-mixed-content; connect-src 'self' https://api.github.com/repos/ fastly-insights.com *.fastly-insights.com *.ethicalads.io https://api.pwnedpasswords.com https://2p66nmmycsj3.statuspage.io; default-src 'none'; font-src 'self' fonts.gstatic.com; form-action 'self'; frame-ancestors 'none'; frame-src 'none'; img-src 'self' https://warehouse-camo.ingress.cmh1.psfhosted.org/ www.google-analytics.com *.fastly-insights.com *.ethicalads.io; script-src 'self' www.googletagmanager.com www.google-analytics.com *.fastly-insights.com *.ethicalads.io 'sha256-U3hKDidudIaxBDEzwGJApJgPEf2mWk6cfMWghrAa6i0='; style-src 'self' fonts.googleapis.com *.ethicalads.io 'sha256-2YHqZokjiizkHi1Zt+6ar0XJ0OeEy/egBnlm+MDMtrM=' 'sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU='; worker-src *.fastly-insights.com
content-type: text/html; charset=UTF-8
location: https://files.pythonhosted.org/packages/source/i/idna/idna-2.8.tar.gz
referrer-policy: origin-when-cross-origin
server: nginx/1.13.9
accept-ranges: bytes
date: Wed, 27 Apr 2022 14:03:23 GMT
x-served-by: cache-iad-kiad7000035-IAD, cache-bur-kbur8200040-BUR
x-cache: MISS, MISS
x-cache-hits: 0, 0
x-timer: S1651068204.829629,VS0,VE80
vary: Accept-Encoding
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-frame-options: deny
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
content-length: 266

HTTP/1.1 200 Connection established

HTTP/2 302 
cache-control: max-age=604800, public
content-type: application/octet-stream
location: https://files.pythonhosted.org/packages/ad/13/eb56951b6f7950cadb579ca166e448ba77f9d24efc03edd7e55fa57d04b7/idna-2.8.tar.gz
server: nginx/1.13.9
accept-ranges: bytes
date: Wed, 27 Apr 2022 14:03:25 GMT
age: 105230
x-served-by: cache-iad-kjyo7100073-IAD, cache-bur-kbur8200085-BUR
x-cache: HIT, HIT
x-cache-hits: 1, 1
x-timer: S1651068206.968264,VS0,VE1
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-frame-options: deny
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
x-robots-header: noindex
content-length: 0

HTTP/2 200 
last-modified: Wed, 26 Feb 2020 17:58:15 GMT
etag: "2e9ae0b4a0b26d1747c6127cdb060bc1"
content-type: binary/octet-stream
x-goog-hash: crc32c=+Bp0RQ==
x-goog-hash: md5=LprgtKCybRdHxhJ82wYLwQ==
server: UploadServer
cache-control: max-age=365000000, immutable, public
accept-ranges: bytes
date: Wed, 27 Apr 2022 14:03:26 GMT
age: 712670
x-served-by: cache-sea4422-SEA, cache-bur-kbur8200085-BUR
x-cache: HIT, HIT
x-cache-hits: 1, 1
x-timer: S1651068207.932473,VS0,VE1
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-frame-options: deny
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
x-robots-header: noindex
content-length: 174481

There is indeed a redirect code here, but it doesn't seem to be 5 times

I found that c0ea3dcb750aed089cfed8d3988e3aef79b70d635cfd42c87db147e860647de4--libvirt-python-6.10.0.tar.gz is indeed downloaded,I deleted it and ran  “brew install virt-manager --debug 2>&1 | tee brew.log” again,it does prompt a redirection error when downloading this file,the output of the second half is as follows:



==> Downloading https://ghcr.io/v2/homebrew/core/vte3/blobs/sha256:ad7a7e195ce43afdf30eff81f83e606b79e78db8e6ed39f0370ef34ebeeb39e4
/usr/local/Homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/3.4.8-21-g0495735\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 10.15.7\)\ curl/7.64.1 --header Accept-Language:\ en --retry 3 --location --silent --head --request GET https://libvirt.org/sources/python/libvirt-python-6.10.0.tar.gz
Already downloaded: /Users/jason/Library/Caches/Homebrew/downloads/d1999493ef40c8221b8eff8d1069d3965f72a8fd6481d3b9701f4fb9344aa007--vte3--0.64.2.catalina.bottle.tar.gz
/usr/local/Homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/3.4.8-21-g0495735\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 10.15.7\)\ curl/7.64.1 --header Accept-Language:\ en --retry 3 --location --head https://libvirt.org/sources/python/libvirt-python-6.10.0.tar.gz
==> Downloading https://libvirt.org/sources/python/libvirt-python-6.10.0.tar.gz
/usr/local/Homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/3.4.8-21-g0495735\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 10.15.7\)\ curl/7.64.1 --header Accept-Language:\ en --fail --progress-bar --verbose --silent --retry 3 --location --remote-time --output /Users/jason/Library/Caches/Homebrew/downloads/c0ea3dcb750aed089cfed8d3988e3aef79b70d635cfd42c87db147e860647de4--libvirt-python-6.10.0.tar.gz.incomplete https://libvirt.org/sources/python/libvirt-python-6.10.0.tar.gz
* Uses proxy env variable https_proxy == 'http://0.0.0.0:1087'
*   Trying 0.0.0.0...
* TCP_NODELAY set
* Connected to 0.0.0.0 (127.0.0.1) port 1087 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to libvirt.org:443
> CONNECT libvirt.org:443 HTTP/1.1
> Host: libvirt.org:443
> User-Agent: Homebrew/3.4.8-21-g0495735 (Macintosh; Intel Mac OS X 10.15.7) curl/7.64.1
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [225 bytes data]
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [93 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [4309 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [589 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=libvirt.org
*  start date: Feb 19 00:31:19 2022 GMT
*  expire date: May 20 00:31:18 2022 GMT
*  subjectAltName: host "libvirt.org" matched cert's "libvirt.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
> GET /sources/python/libvirt-python-6.10.0.tar.gz HTTP/1.1
> Host: libvirt.org
> User-Agent: Homebrew/3.4.8-21-g0495735 (Macintosh; Intel Mac OS X 10.15.7) curl/7.64.1
> Accept: */*
> Accept-Language: en
> 
< HTTP/1.1 200 OK
< Date: Wed, 27 Apr 2022 14:15:46 GMT
< Server: Apache/2.2.15 (CentOS)
< Last-Modified: Tue, 01 Dec 2020 08:58:26 GMT
< ETag: "6a0138-33e57-5b56354675080"
< Accept-Ranges: bytes
< Content-Length: 212567
< Connection: close
< Content-Type: application/x-gzip
< 
{ [16384 bytes data]
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):
} [2 bytes data]
/usr/local/Homebrew/Library/Homebrew/shims/shared/curl --disable --cookie /dev/null --globoff --show-error --user-agent Homebrew/3.4.8-21-g0495735\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 10.15.7\)\ curl/7.64.1 --header Accept-Language:\ en --retry 3 --location --silent --head --request GET https://pypi.io/packages/source/i/idna/idna-2.8.tar.gz
Error: Too many redirects (max = 5)
/usr/local/Homebrew/Library/Homebrew/utils/curl.rb:410:in `parse_curl_output'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:469:in `resolve_url_basename_time_file_size'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:453:in `resolved_url_and_basename'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:320:in `resolved_basename'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:304:in `cached_location'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:277:in `temporary_path'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:388:in `fetch'
/usr/local/Homebrew/Library/Homebrew/resource.rb:147:in `fetch'
/usr/local/Homebrew/Library/Homebrew/formula_installer.rb:1183:in `each'
/usr/local/Homebrew/Library/Homebrew/formula_installer.rb:1183:in `fetch'
/usr/local/Homebrew/Library/Homebrew/install.rb:305:in `block in install_formulae'
/usr/local/Homebrew/Library/Homebrew/install.rb:278:in `map'
/usr/local/Homebrew/Library/Homebrew/install.rb:278:in `install_formulae'
/usr/local/Homebrew/Library/Homebrew/cmd/install.rb:218:in `install'
/usr/local/Homebrew/Library/Homebrew/brew.rb:110:in `<main>'
jason@osx ~ % 

So far, the redirect error can be caused by downloading the idna-2.8.tar.gz. But when I run it with brew, it only appears, I can also download this file by entering this url directly in the browser.

Can I download manually this idna-2.8.tar.gz

[Bo98]

Can I download manually this idna-2.8.tar.gz

Don't worry about that. The limit is now 25 so just update brew.

I think the issue might have been HTTP/1.1 -> HTTP/2 upgrades counting as a redirect. We'll remember that for the future.

[wykeinstein]

Alright, thank you very much.
I will try it after upgrade the brew

[samford]

I think the issue might have been HTTP/1.1 -> HTTP/2 upgrades counting as a redirect.

Yes, exactly this. The "Too many redirects" error message is misleading because normally we don't use a redirection limit (i.e., --max-redirs) and max_iterations simply limits the number of responses that are parsed in the output. The while loop in #parse_curl_output looks for an HTTP status line and the \r\n\r\n separator, so the error is more like "More HTTP responses than we care to parse".

For the sake of making the error message palatable, I baked in the assumption that intermediate HTTP responses would always be redirections but clearly that won't always be the case. Since the limit is higher now and this error message is less likely to appear under normal circumstances, there may be something to be said for using something like "Too many iterations while parsing HTTP responses (max = 25)". This wouldn't mean much to the user but avoiding "redirection" language would be more accurate and make it clear exactly where the problem is coming from (since parsing happens after curl has finished making requests).

[wykeinstein]

After I update brew to 3.4.9-6-g58aa2e1 version，The error with the redirect code doesn't reappear and I installed virt-manager successfully.
Thanks to all of you for helping me solve the problem