X-Amz-Date header missing or set multiple times

[MaxRink]

We have a Django Application ( Netbox) which uses the Django backend for S3.
This doesnt set headers but instead puts things into the URL which leads to this error:

X-Amz-Date header missing or set multiple times: &{GET /netbox-test/devicetype-images/2021-03-02_13_28_22-ASR-9910.jpg_700800.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=OMKK45MUP3A21NUFBAMO%2F20210901%2Ffx%2Fs3%2Faws4_request&X-Amz-Date=20210901T194621Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=8cd5167be8363d44b437e2ddb9b09dcb7e2622375e3122d18c370ca7b7ff1a4b HTTP/1.1 1 1 map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-GB,en-US;q=0.9,en;q=0.8,de;q=0.7] Cookie:[CONSENTMGR=consent:true|ts:1622119709217; wt_ttv2_s_196380495960676=9999; SAPWP_active=1; wt3_sid=%3B943593591056212%3B700378032410688%3B896369902135262; utag_main=v_id:017b9e117cb60019cfa8f2fe36a202069008406101788$_sn:2$_ss:0$_st:1630497112174$ses_id:1630495283764%3Bexp-session$_pn:8%3Bexp-session; wt3_eid=%3B943593591056212%7C2162972062014115676%232163049134992127594%3B700378032410688%7C2162998334043534258%232163049531157297740%3B896369902135262%7C2162998334087666506%232163049531287865764; wt_rla=943593591056212%2C20%2C1630491121687%3B700378032410688%2C5%2C1630495284161%3B896369902135262%2C4%2C1630495284251; _oauth2_proxy=tNL3RjEJLxE0V-3hZk0_BQNt_kCNCCTQJHs7zeNpPEDRAiwa-QoWtQysdpspA_MLS5L9DAJQv7O8WDJ0I9lQzyQYoGfgoE142AyJsyVTVsjLuFIxyQW1JhVyuptBhN3lkUArNHlN15826fVZSbP2KXOGOb3UQj4LsOQmCVunaxeyeETs3kTqbigGz_SAi1Q7tgs8VZovKJUgeUovCXwla3Gjh8ZfD25GptALAyuhmlZJuxyKmDZ9qeSlr44IC0liz6rAsZ6c5SoB3Mg8dLcxR3wLjZgKrbhF9UAwX8E3p_HFXSZJqoYkLJPj7bd7DlTjLIcysfFq__laq7nfeHk805w2ZsrKVOYpF1UUL6sx6i-DJx-Kc0LMqXjb0KfOkxnjNhwPNwF_EBrEteyKZISVOEZTYiK60FJas6GIWgs9paZq8Vvn1wV8--NF9AFLouPrL78hAX0DKVgkqOhfcUYrazs_HHfXqKv4T_7vjUOqZmHO3YBdcmtCN0CfhRJejAUw1bpW-aH4k9FkI8x-imuq68cHUPu9xEQfOVBmZ4ta69JKqcW6uDl20X89VWMt0MDs5_jJnQQuYloEhwsYFQ3RuBTbnDgEjHhP1pLVLth8UW65KGNPlhTwypYzV-hnpvelvP_zELqEDjwd26dvZmBVJA4f5pVTCSefbjxOfDLopmsXKd5pj7hRqgC_pg7vBeWk-sesA5V1Ew-Io9IwzIBT9TGrtsssxSyz5U4QLsYI1YQw4cqU4rGf8NinGecBiSsUM91pTbKdsv_fIunzrsBEUQLmSnUnf2SESCIR3FU-nJcv0cMyTI1US2fBScpPB5h6KLd6PHg2fJf_CJvSdiPt-7PXdMjMeBfI37SOBkpQ-RYz_ShbzxBS54ZkbR_gHjqlIAPYH4L9DTPNcNwCCMUjTInYHfwaBgWbtocnbUh1W5l2XKLaUclcYHyObS2wchcf6PtYH4AGLC_K_Lov2eRuwgg6z3KTNg2Xr8phZ5fNry273qdf-W-ThMZAZIkl_GjsbwqOMpJzmKMzeXFiAxV2zWpLevVnA0HtsJdrjFgpvEcSIRjDf79f2noiY-EvpXHk-3Yy-dkip4ngA_4mzjk42grIsqQf0T4qpmzU9WOsjpcASKVDRq1Kg-FsZHxALOS6x06fXJkR_V6M7uEBZZuy_cL3r1XfDRcAMHaDpv5dCmYO5SIdTOEhAAuFPqCsIfussi77fqXmhG9RzjW8G2Ap_iX_LJbbqdaOuYFTIajPeFCjhmzBEvEM5ULl8qDKFYTaFwNoRinwjzrQblwK71Iv76e3CzEQQH5J9Xh3CjXxU6kkYynHEAfQgy9al6lFHEFGLKAqwvK6WZTmrx1yqvdhICcsu2s4K4Y-R0mOHIYYlz_zzNpGHlBGRZPFfiY0bx1UPLEbrrmj6xdXJNPYZim3n1H3z6ocJ_9J4v_1DohNlCdfIkvTm9i9x0n--1Ouy_QUy2TyUsD-6m0l_REuJePOILygxQ5T9RuRf3nQsl3P_GFQ2z2YrmCLpp7sAx61n5pldKPYzuLLgeoT7GizyQCmZW8d5_2R9mFGoZFVUVA9wA1pA3oEyj4rtKDvqp061pVamaknE9p427kJA5mlckRj5RZxMnKLyYUFA6HRO-rlEfT8xlDxMaNM8JAHGr4QyZO5GaXF5D9-6E_7RTVvM2v7Jz12lXYAmK6TxuV0XwRdCNYPjJfG-kXghXmPf1unNuMQpxTyBr6Nohd4CbJchuN95B1kz8_9KZTrDEe8zlSGIzAFVMF7jPPrVrM93jpaTbKphQ567zuQzVTUX0g4cRpwktJZzpefgfXbSSXIeUMzwAZ9lwBrUuOtsXqdGrTpd8iPma85FPwHvgQ8Fv0seco_AT-gZERjtWZJS5MdW01oMYbvHmDKTGv7gGg6ignjeYXw4vuWEooFQVT_sqQGDyRHf9X0-foqO0dIDFRiB0E2f1_Lu2VIBFNY8JS79WD0kWxo7cSIvyLniG1F7Ky0GnQ3V_XhlNxy8HdNoBOjhSTAt3vYMI_hP8IjNgd9K_DT9dIqTcd5kKMwsrLpEjKU9ygw99oeorZjohFblnQ1Yrs9Df-t3nR6kC8mCrIWDTMCEji7_Gr27VPHdYjXcFGbVugUe4gUkX3HBQUgpOQxKkJYBih2APBO2oTjuWFWVs3HKuTmfpOUe3RpMyC37khbRTx8GA6QPwbH1dtKPA0-mCXQ0k7P0XdizKM9OPoWJGsy3cKVy7scq9YkGRtPg0oCF89O21M6OFXK1LSzD36ZB5RJYJKIiauFOyQa3IupRYbPg85xm2enSk_lr9tG8dO1aBG30bHDq_X_y1SLzuG4Ax8hznFygFe10U4vopZ1X1MQC4tV1gVugmnIZQ2EXg_ScZr9kJA2ffWHwu2CsWBLPYK3HK4s4PukeOkQJVCUOoD3nZ3lgZLyu4f3w5a41zw7-6SQ8xc86FuPs_k6ZzNnCPgf0uNa6k95JelzPX-Y2LUBWkKTLl7jYVq4sJDrHmZ5Gy5JhO-EkGYxoUiYEy5BNQYKsJ9EBLa3g2k6gG2J4J7dAt-Tgi_cz6GwhHeTzxlbuuUeBgqt3ME_VrAfzJhqHwp87XW52X3ua03vhFLzckZnDAt2D1rdc2kET4gCV837K9AsR-pHzrgO-91LPw9xVq469C3UBPOMSRUWIZ-T-qG-ovkpN97z8SGwCuhZEfOnq_bN20JciT-THbC0szzLBE5kpNe0VPZ4HoOn_tjyMhCLI8WVn4xjoiqzEvc3OOOnkrinU9qUPRoNMXEvQBMdDAoswumhjwYqstFJ-qoAiduBFYpoVvB31iCTbyxfr2OaOWGA4CTaFdVXnU8XWgrDUkvPsF4xC4VgXbEs7sDAM1tOv0_E6Xc0yIqX4m9_lSJpfqcT-x0xTWfEUA6nUZUo49mIlc5HdAq_rEBvCWuGwuw_gaE_CNl4DJEIcnD1PXlp-kCyOktW4WZtGJ7GO3FZaLScVL_JRqdLlX1_jP-ExT9Fy15iKFrfjwzU80lrYAioOfzHw-rgWxxEWJu9DvqN0Qeu0ws59jnZjBCXhbnUle1SV5vTuVqv970vT4QZlkUgm9DtChw77q_voMwxg7rcx1FIKhzlh_jDqJgPvUis9Kb2K_O5k7OCm0iTPxYmBKRLflsOVgviBfudQMnu39EMi1J_O-2CTJBYSbQifVlvHPNdUmJh0CaqJFmhJ60UvLYyjf27A0mRwbe_zxUy81Aywr5pyN0JW6ZZEDU14nFUN1MyNZ3GvJEoWGokym1dbulsNkBPm5sgyAzyui7Icf-f5M9BkePbWoJMdMI7fmIjnL0suXxnR5KPeSTHclvzc-qIEhp-TlJ-da8GOouKzMC2s2bIRzTqkpSEkl6Gh7py369imYRl2AslaKVZCghZ3P_MYtD0l1ig1MnLcDwCqgH_jLG2fAOh7cZXA0slpF37pKNFIu3pZ1XS1a5nce3Gvzu4w--Q5SJbyD99hgjRJgY3Id7oZNGgvnQqOokPc2cnnRYvw9fke-8b_QCZWj_bfVIO376P|1630509542|9Kw5U7pvkklkmlVsk8pX2txQTPjl-H9TBpBeR-5hy8Y=; csrftoken=4jGnRUwzXAORCKRbEduMCRHBsStKM6l5Y3hQCFffudRVgKcqAnMP813Q8QQioYzP; sessionid=mqz6u2cu16xowpwk0kte09g60ksiad56] Referer:[https://netbox.hb.das-schiff.telekom.de/dcim/device-types/25/] Sec-Ch-Ua:[" Not A;Brand";v="99", "Chromium";v="92"] Sec-Ch-Ua-Mobile:[?0] Sec-Fetch-Dest:[document] Sec-Fetch-Mode:[navigate] Sec-Fetch-Site:[same-origin] Sec-Fetch-User:[?1] Upgrade-Insecure-Requests:[1] User-Agent:[Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36] X-Forwarded-For:[10.27.50.159] X-Forwarded-Host:[netbox.hb.das-schiff.telekom.de] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Real-Ip:[10.27.50.159] X-Request-Id:[0ac451fcb2b561c5bbfcc1aabafdbdfc] X-Scheme:[https]] {} <nil> 0 [] false netbox.hb.das-schiff.telekom.de map[] map[] <nil> map[] 192.168.90.70:37558 /netbox-test/devicetype-images/2021-03-02_13_28_22-ASR-9910.jpg_700800.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=OMKK45MUP3A21NUFBAMO%2F20210901%2Ffx%2Fs3%2Faws4_request&X-Amz-Date=20210901T194621Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=8cd5167be8363d44b437e2ddb9b09dcb7e2622375e3122d18c370ca7b7ff1a4b <nil> <nil> <nil> 0xc000396700}

[Kriechi]

I think you are referring to pre-signed URLs / query parameters as described in https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html

Unfortunately this is currently not implemented - only headers are considered at the moment.

I'd be happy to review and merge a PR!

[MaxRink]

Will see if i find time for it next week, shouldnt be too hard, as you current code looks quite expandable for this on the first glance

[333miiko]

@MaxRink @Kriechi any plans to add this?

My error seems to be different

AccessDenied: There were headers present in the request which were not signed
HeadersNotSigned: 'x-amz-request-payer, x-amz-user-agent'

Unfortunately I don't know golang :(

[scienced]

I ran into the same error using the GoodSync client
error="X-Amz-Date header missing or set multiple times: &{GET ...