sam build with UnicodeDecodeError: 'utf-8' codec can't decode byte 0x91 in position 565: invalid start byte

[mikezang]

Description

I installed SAM CLI, version 1.2.0, and run sam build and get error as below:

.UnicodeDecodeError: 'utf-8' codec can't decode byte 0x91 in position 565: invalid start byte

Steps to reproduce

command: sam build

Observed result

Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
No Parameters detected in the template
Building function 'HelloWorldFunction'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r C:\zang\sam\sam-app\hello_world\requirements.txt --dest C:\Users\120000843\AppData\Local\Temp\tmpolmpbuso
PythonPipBuilder:ResolveDependencies raised unhandled exception
Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflow.py", line 269, in run
action.execute()
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\actions.py", line 42, in execute
requirements_path=self.manifest_path,
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 137, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 204, in build_site_packages
wheels, packages_without_wheels = self._download_dependencies(scratch_directory, requirements_filepath)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 228, in _download_dependencies
deps = self._download_all_dependencies(requirements_filename, directory)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 310, in _download_all_dependencies
self._pip.download_all_dependencies(requirements_filename, directory)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 595, in download_all_dependencies
error = err.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x91 in position 565: invalid start byte

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 38944, 'exitReason': '
WorkflowUnknownError', 'exitCode': 1, 'requestId': 'b033b571-4980-4874-8ed7-f739050281c9', 'installationId': 'a37859ad-7d82-4995-9f9b-0d92263f0318', 'sessionId': 'e2f3e2d9-ab99-42e
a-a798-f516e13c5ea1', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '1.2.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Max retries exceeded with url: /metrics (Caused by NewConnectionError('<urllib3.connec
tion.HTTPSConnection object at 0x0000000004222FC8>: Failed to establish a new connection: [WinError 10061] 対象のコンピューターによって拒否されたため、接続できませんでした。'))
Error: PythonPipBuilder:ResolveDependencies - 'utf-8' codec can't decode byte 0x91 in position 565: invalid start byte

Expected result

Successful to build

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Windows7
2. sam --version: SAM CLI, version 1.2.0

[mikezang]

Anyone can help me?

[wchengru]

Looks like your requirement.txt is not UTF-8 encoding, and pip requirement file needs to be UTF-8.
I can reproduce this issue by adding the ANSI character "0x91" into the requirement.txt file.
Run:

pip download -r ./sam-test-non-utf8/hello_world/requirements.txt 

and get:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x91 in position 2: invalid start byte

My suggestion is converting the requirement file to UTF-8 and it will resolve this pip issue.

[mikezang]

my hello_world/requirements.txt only inlucde "requests", I think they are the same both utf-8 or shift-jis.
Anyway, I followed your suggestion to save that file in utf-8:

then run sam build --debug again, I got the same result as below:

[mikezang]

I am behind of Proxy, after run pip download --trusted-host pypi.org --trusted-host files.pythonhosted.org --proxy http://my-proxy.com:80 -r C:\zang\sam\sam-app\hello_world\requirements.txt --dest C:\Users\120000843\AppData\Local\Temp\tmp5fnj51h8, I got result what I need:

The problems is how can I run sam build with --proxy and --trusted options in pip download, do you have any idea?

[mikezang]

After proxy and trusted as below to %APPDATA%\pip\pip.ini,

[global]
trusted-host = pypi.python.org
pypi.org
files.pythonhosted.org
proxy = http://my-proxy.com:80
run sam build is successful.

[wchengru]

Output of the pip or other packages should be improved to give more informative installation errors. Related issue for this: aws/aws-lambda-builders#197