Build failed - Error: PythonPipBuilder:ResolveDependencies

[fniko]

Description

Trying to create Python Lambda function from PyCharm not working

Steps to reproduce

Invoke Lambda creation using "Create new AWS Lambda..."
Runtime: Python 3.7
requirements.txt: https://pastebin.com/HpU1it63
Run using strict template file:
template.yml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An AWS Serverless Specification template describing your function.
Resources:
  test3:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: order.create_order
      Runtime: python3.7
      CodeUri: .
      Description: ''
      MemorySize: 128
      Timeout: 300
      Role: 'arn:aws:iam::123456789:role/service-role/test-role'
      VpcConfig:
        SecurityGroupIds:
          - sg-62e1c42f
        SubnetIds:
          - subnet-26e9c26e
          - subnet-68a3c632
          - subnet-b8edb5de

Observed result

Used graphical create, not CLI, error:

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {msgpack-python==0.5.6(sdist), pyrsistent==0.15.7(sdist), simplejson==3.17.0(wheel), pyyaml==5.3(sdist)}

Full log: https://pastebin.com/asgCa9D4

Expected result

Successful build

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: macOS Mojave 10.14.6
SAM: SAM CLI, version 0.43.0
P: Python 3.7.6
IDE: PyCharm 2019.3
AWS: aws-cli/2.0.1 Python/3.7.4 Darwin/18.7.0 botocore/2.0.0dev5
used wheel: wheel 0.34.2
used pip: pip 19.0.3

[fniko]

I solved this using sam build --use-container, however this works only when template.yml file is present.
Using the graphical form does not allow to insert --use-container and it fails, so I do not count this as a solution, but workaround instead...
// Also since I am unable to build app locally in my IDE, I am unable to run/debug it, so it makes no sense to use this workaround.

[c2tarun]

Are you using AWS Toolkit for Pycharm?

[dperconti]

I'm seeing something similar:

$ sam build
Building resource 'PaymentsDataStreamFunction'
Running PythonPipBuilder:ResolveDependencies

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {pyyaml==5.3.1(sdist), pyrsistent==0.15.7(sdist)}

Both pyyaml & pyrsistent are inside my requirements.txt file, which have been installed inside a virtualenv.

[sriram-mv]

Do you have wheel installed? if not sam build --use-container may be the best bet.

[dperconti]

wheel is installed. --use-container also fails with a similar error but with different deps. I tried creating a brand new project which worked fine. So, it's probably something within my project that is messed up.

[fniko]

@c2tarun yes, I am using this plugin

[heitorlessa]

Same here - Used another dep that depends on jsonschema and bumped into this. Can't use --use-container as I have a shared dep two directories up. For that to work I'd need to copy the wheel to every service I have, or have a way to mount a volume during sam build.

Thanks for opening the issue, I'll spend a couple of days rewriting the lib and build a wheel from that instead

[emulvihill]

I'm getting same issue when I use sam build --use-container (Windows, Python 3.6, SAM CLI version 0.47.0):
Build Failed Error: PythonPipBuilder:ResolveDependencies - {tensorflow==1.15.0(wheel), grpcio==1.28.1(wheel)}

[guywilsonjr]

Same. It seems like it fails to build anything with tensorflow or better yet grpcio

[viotti]

My current workaround is to manually add grpcio to the requirements.txt file, but fixed to an older version.

grpcio==1.27.2

It seems to fail with version 1.28.

[deanrabinowitz4]

I'm seeing a similar error with the following in requirements.txt
pyorc==0.2.0

I am running:

sam build --use-container

Output:
Starting Build inside a container
Building resource 'CdcFunction'

Fetching lambci/lambda:build-python3.7 Docker container image......
Mounting /path/to/my/app as /tmp/samcli/source:ro,delegated inside runtime container

Build Failed
Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - {pyorc==0.2.0(wheel)}

wheel is installed. I'm getting the same error when I add psycopg2 to the requirements.
Here is my template.yaml for reference

Globals:
  Function:
    Timeout: 180

Resources:
  CdcFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: lambda/
      Handler: lambda.lambda_handler
      Runtime: python3.7

[mooperd]

+1

[carpnick]

+1

Error: PythonPipBuilder:ResolveDependencies - {pyrsistent==0.16.0(sdist), pyyaml==5.1.2(sdist)}

[sriram-mv]

If your dependencies have C bindings involved, building within the container is the way forward in those cases. Toolkits have an explicit checkbox that can be enabled to allow for all builds to be within a build container.

Closing this issue, please re-open if you feel this doesn't answer your original question.

[etishka]

got the following error started appearing during sam build

Error: PythonPipBuilder:ResolveDependencies - {protobuf==3.17.1(wheel)}

SAM CLI, version 1.20.0
Python 3.8.7
pip 21.1.2

--use-container doesn't solve the problem.

[greensky-code]

UnsatisfiedLinkError: com.amazonaws.kinesisvideo.producer.jni.NativeKinesisVideoProducerJni.getNativeLibraryVersion()Ljava/lang/String
I am following this workspace https://github.com/backdoorcodr/amazon-kinesis-video-streams-producer-sdk-java
Expectation is to run kinesis video stream using webcam in java.
Please help

[leonardishere]

I was running into Error: PythonPipBuilder:ResolveDependencies - {cytoolz==0.11.0(wheel)} with sam 0.45.0. Upgrading sam to latest (1.27.2) solved it

[CGarces]

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {pillow==8.3.1(wheel)}

Solved updating SAM CLI from version 1.22.0 to version 1.29.0

[otmezger]

I'm having the same issue

sam build --use-container --template ./my_template.yaml --debug

produces

Error: PythonPipBuilder:ResolveDependencies - Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-download-ufgf7ow5/fastparquet/

However, installing the requirements.txt with pip in the host machine produces no error.

I also tried to update my sam version from 1.27.2 -> 1.31.0 and I get the same error.

[dslove]

I have the same issue. Tried installing wheel, using --use-container, updating sam to 1.31. None is helpful.

The most weird thing is it was working last week, but suddenly broke today. Nothing has changed at all!

[hoffa]

Reopening since this still seems to be an issue. Potentially related: aws/aws-lambda-builders#280

[virajkanwade]

I think I was able to get little more insight into the issue.

MacOS + pyenv

❯ head -n 1 `which sam`
#!/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/bin/python3.8

so the expectation would be following to work

/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/bin/pip3.8 install wheel

After some digging around did this:
edit /usr/local/Cellar/aws-sam-cli/1.37.0/site-packages/aws_lambda_builders/workflows/python_pip/packager.py line 658

before or after invoke_pip = [self.python_exe, "-c", exec_string] added print(self.python_exe)

to my surprise it was pointing to /Users/XYZ/.pyenv/shims/python3.8

ran /Users/XYZ/.pyenv/shims/pip3.8 install wheel

and then sam build and it worked.

Please check and confirm here if this works for you too.

[AbdulBasitKhaleeq]

same issue for jsonschema.

Build Failed
Error: PythonPipBuilder:ResolveDependencies - {pyrsistent==0.18.1(wheel)}

[croosen]

Same issue with autobahn (although this can happen with any dependency):

Error: PythonPipBuilder:ResolveDependencies - {autobahn==22.3.2(sdist)}

In my case the used version was not compatible (22.3.2). I updated my requirements.txt with:

autobahn==21.2.1

and then run:

pip3 install -r requirements.txt

I found the working version after some searching in the repo of the main package using this dependency but this worked for me.

[maruf897]

@

I solved this using sam build --use-container, however this works only when template.yml file is present. Using the graphical form does not allow to insert --use-container and it fails, so I do not count this as a solution, but workaround instead... // Also since I am unable to build app locally in my IDE, I am unable to run/debug it, so it makes no sense to use this workaround.

how do you test it locally after building with container?

[Leslie-Wong-H]

Just add some extra insights here.

sam project structure as follows:
|- template.yaml
|- samconfig.toml
HelloWorld
--- |- requirements.txt
--- |- mymodule
------- |- setup.py
------- |- setup.cfg
------- |- src
------------|- code
--------------- |- __init__.py
--------------- |- code.py

I am using a local module and declare it as "-e relative/path" at requirements.txt.

The problem is that the relative path declared is very tricky.

When sam build --use-container from the project root, it should be -e mymodule/. Otherwise it would cause PythonPipBuilder:ResolveDependencies, yet -e mymodule/ wouldn't work the same when sam build from the project root.

[armando-herastang]

This fixed the ResolveDependencies error for me pip install wheel

This definitely worked for me. But, could someone explain why is this needed? It would be great if it worked out of the box

[KIVagant]

My set of tricks to make it work on Apple M1 (MacOS 11.6 Big Sur)

brew remove aws-sam-cli # will be installed below using pip

brew install pyenv
# Also you may need to remove an existing python version.
# I had to remove and reinstall it back later after all other commands to make things work
# brew remove [email protected]
pyenv install 3.7.13   # < 3.7 is currently required by AWS SAM
pyenv global 3.7.13

vim ~/.zshrc || vim ~/.bashrc

### add these lines to the *rc file:

  # >> PYENV
  export PYENV_ROOT="$HOME/.pyenv"
  export PATH="$PYENV_ROOT/bin:$PATH"
  eval "$(pyenv init --path)"
  # << PYENV

#save and close the file

#open a new terminal window

pip install aws-sam-cli wheel

# you may also need this if you have Podman, even if you build locally:
# pip install paramiko

Check versions:

python -V && pip -V && sam --version

# Python 3.7.13
# pip 22.0.4 from /Users/yevhen.hlotov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/pip (python 3.7)
# SAM CLI, version 1.53.0

Build the project with sam build and it should work now.
Although, it still cannot build in containers when using Podman due to the separate issue.

[austinrbrown01]

To anybody who sees this in the future, the I was having the same issue as this thread (and like others, I was able to resolve using sam build --use-container. It turns out that I didn't have a virtual environment set up in my project root, which makes sense why this would create dependency issues.

I only figured this out because my deployment package became too large to deploy, and I realized it was bundling a ton of packages I didn't need. This led me to realize I didn't have an virtual env configured.

[LoganB99]

I am still seeing this issue with sam build --use-container. Using podman on macOS (tried PKG, homebrew, and now pyenv. Using DOCKER_HOST. sam local invoke and sam local start-api only work with ---skip-pull-image. sam build without use-container works

Build Failed
Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - Could not satisfy the requirement: requests

trying the hello world example

[erikfinnman]

On the surface, my problem seemed to be the same as many others here. But it turns out that my issue was an expired token to our CodeArtifact repository (which we use for all our Python dependencies). This became evident when using the --debug option and seeing that sam build was unable to download any dependencies.

[sergiors]

I have the same

Build Failed
 Running PythonPipBuilder:ResolveDependencies
Error: PythonPipBuilder:ResolveDependencies - {pendulum==2.0.5(sdist)}
make: *** [build] Error 1

[VctrEman]

I have the same, but for another package: lightgbm 4.2.0

Final incompatible: {lightgbm==4.1.0(wheel)}
Final missing wheels: {lightgbm==4.1.0(sdist)}
PythonPipBuilder:ResolveDependencies failed

I had to downgrade my lgbm version to 3.3.5 and it the build worked, testing locally too.