Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python builder can give a hint to build on linux system, whenever a dependency fails because of its C (native) dependency #208

Open
mndeveci opened this issue Oct 30, 2020 · 1 comment

Comments

@mndeveci
Copy link
Contributor

Steps to reproduce the issue:
Whenever sam build command fails for certain dependencies which requires to be build on linux environment (because of C dependencies of that package), output logs can contain a hint for user to build their functions on a linux environment (probably using --use-container flag)

Steps to reproduce the issue:

  1. Init a project (ex: HelloWorld) which uses python3.8
  2. Update requirements.txt file inside the function, and add aws-xray-sdk as a dependency
  3. Try to build function on Mac or Windows by running sam build

Observed result:
Build fails with a dependency error;

C:\agent2_work\142\s\Assets\events\Post>sam build --template "C:\agent2_work\142\s\Assets\events\Post/Assets_Events_POST-build.yaml" --region us-west-2 --debug
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='AssetsEventsPOST' and CodeUri='.'
No Parameters detected in the template
Building function 'AssetsEventsPOST'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r C:\agent2_work\142\s\Assets\events\Post\requirements.txt --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i
Full dependency closure: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), wrapt==1.12.1(sdist), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(sdist), importlib-metadata==2.0.0(wheel)}
initial compatible: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), importlib-metadata==2.0.0(wheel)}
initial incompatible: {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
Downloading missing wheels: {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i future==0.18.2
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp37m --dest C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i wrapt==1.12.1
compatible wheels after second download pass: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), importlib-metadata==2.0.0(wheel)}
Build missing wheels from sdists (C compiling True): {future==0.18.2(sdist), wrapt==1.12.1(sdist)}
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\future-0.18.2.tar.gz
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\wrapt-1.12.1.tar.gz
compatible after building wheels (no C compiling): {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Build missing wheels from sdists (C compiling False): {wrapt==1.12.1(sdist)}
calling pip wheel --no-deps --wheel-dir C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i C:\Users\dummyUser\AppData\Local\Temp\tmp85csqu6i\wrapt-1.12.1.tar.gz
compatible after building wheels (C compiling): {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Final compatible: {python-dateutil==2.8.1(wheel), jsonpickle==1.4.1(wheel), six==1.15.0(wheel), zipp==3.3.0(wheel), botocore==1.18.15(wheel), aws-xray-sdk==2.6.0(wheel), jmespath==0.10.0(wheel), urllib3==1.25.10(wheel), future==0.18.2(wheel), importlib-metadata==2.0.0(wheel)}
Final incompatible: {wrapt==1.12.1(wheel)}
Final missing wheels: {wrapt==1.12.1(sdist)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\actions.py", line 42, in execute
requirements_path=self.manifest_path,
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 137, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\packager.py", line 201, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {wrapt==1.12.1(sdist)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflow.py", line 269, in run
action.execute()
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\python_pip\actions.py", line 45, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {wrapt==1.12.1(sdist)}

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': 'us-west-2', 'commandName': 'sam build', 'duration': 20257, 'exitReason': 'WorkflowFailedError', 'exitCode': 1, 'requestId': 'c5551ae9-caea-4b93-9c48-d07fa2cbc8e3', 'installationId': 'b7024037-b875-42d8-8e62-2abac9896f08', 'sessionId': 'd38fda56-1b47-4cf4-8840-65a6966432b4', 'executionEnvironment': 'CLI', 'pyversion': '3.7.6', 'samcliVersion': '0.53.0'}}]}
Telemetry response: 200
Error: PythonPipBuilder:ResolveDependencies - {wrapt==1.12.1(sdist)}

Expected result:
If the build failure is because of C dependency, it may contain more information for user which will suggest them to run build on a linux environment, probably using --use-container

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
This only happens on non linux environments (Mac and Windows).

Related issues

@sriram-mv
Copy link
Contributor

This should also tell the user if they have "wheel" installed or not, since they can be building a dependency which only has an sdist distribution on pypi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants