Python code failing type checking

[rix0rrr]

Reported in another issue

@dmartin-gh

I am also experiencing the interface issues described above. We're looking to adopt the CDK for our main platform and as a mostly Python/C++ shop the Python CDK seemed like a great choice for us. I didn't make it very far into my first sample app to test things out before mypy started throwing errors.

from aws_cdk.core import Construct, Stack
from aws_cdk import aws_ec2 as ec2

class MyEC2Stack(Stack):

    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        ec2.Instance(self, 'my-app',
            instance_type=ec2.InstanceType('m5.4xlarge'),
            machine_image=ec2.MachineImage.generic_linux({
                'us-east-1': 'ami-0affd4508a5d2481b', # CentOS 7 (x86_64) - with Updates HVM
            }),
            vpc=ec2.Vpc(self, "my-app-vpc"))

I would love to see this fixed so we can start building out our application stack with confidence that we're passing the right level of constructs around!

Is this still a problem today? I have been trying to reproduce this (with CDK v2, though), and am not getting the MyPy error you report here... if you still happen to have the error, maybe this is because of your specific MyMy version or configuration (in which case, I would like to know what these are so I can reproduce).

Indeed it is still a problem. For example try loading up apigw-http-api-lambda-dynamodb-python-cdk in vscode, then pyright reports for

        # Create the Lambda function to receive the request
        api_hanlder = lambda_.Function(
            ...
        )

        ...

        # Create API Gateway
        apigw_.LambdaRestApi(
           ...
            handler=api_hanlder,  # Pyright error: Argument of type "Function" cannot be assigned to parameter "handler" of type "IFunction"
        )

typecheck error

Argument of type "Function" cannot be assigned to parameter "handler" of type "IFunction" in function "__init__"
  "Function" is incompatible with protocol "IFunction"
    "IFunction" is incompatible with "Function"
    "IFunction" is incompatible with "Function"
    "IFunction" is incompatible with "Function"
    "IFunction" is incompatible with "Function"
    "IFunction" is incompatible with "Function"
    "IFunction" is incompatible with "Function"
    "IFunction" is incompatible with "Function"Pylance[reportArgumentType](https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportArgumentType)
(variable) api_hanlder: Function

as shown in this screenshot

That project uses aws-cdk-lib==2.77.0 however the same issue exists after updating to recent aws-cdk-lib==2.143.1. On my machine it happens with the following configuration

pyright 1.1.364
pylance 2024.5.103
aws-cdk-lib 2.143.1
python 3.11.2
vscode 1.89.1

Originally posted by @mariogalic in #1919 (comment)

[polothy]

Also related to this issue:

• MyPy throwing type errors when it shouldn't #3931 (another user experience report)
• feat(python): support inheritance-based interface implementation #3350 (perhaps a fix?)

I'm also still seeing this issue (#2877) in a JetBrains IDE (Intellij IDEA Ultimate with Python plugin). A comment in the issue near the end says it's fixed, but I see the same Expected type 'str | str | None', got '() -> str | str' instead error in my IDE today using CDK v2.144.0.

These two typing issues combined make Python experience really rough because your IDE thinks you have a lot of invalid types and sometimes its hard to see which typing issue is an actual problem unless you run cdk synth/deploy.

[mikewrighton]

I took a look at apigw-http-api-lambda-dynamodb-python-cdk and was able to root cause at least one issue causing these type checker errors.

When you implement an interface in TS, the method names have to match, but the method argument names do not, as long as the argument types are in the correct order. In Python however, when you implement a Protocol (which is how we do interfaces), the method argument names do have to match. So when our TS library has things like IPrincipal.addToPrincipalPolicy(statement: PolicyStatement) and an implementation of PrincipalBase.addToPrincipalPolicy(_statement: PolicyStatement) the former generates an IPrincipal Protocol in python, and the latter generates a class that's incompatible, and we get the PyRight error:

...
        "AnyPrincipal" is incompatible with protocol "IPrincipal"
          "IPrincipal" is incompatible with "AnyPrincipal"
          "IPrincipal" is incompatible with "AnyPrincipal"
          "IPrincipal" is incompatible with "AnyPrincipal"
          "add_to_principal_policy" is an incompatible type

The suggested fix here is to always use the method argument names in the base (Protocol) class, when generating subclasses. In the example above, PrincipalBase.addToPrincipalPolicy() would use the method argument statement instead of what's in the TS code, _statement. I didn't get as far as implementation but hopefully this context is useful.

[adriandelgg]

I have this issue too in Intellij IDEA Ultimate w/ Python. Super annoying. I don't get the error in VS Code.

[dytyniuk]

I used typing.cast(aws_iam.IPricipal, principal) as a muffler, but got fed up with it. So, upvote.

[adriandelgg]

I used typing.cast(aws_iam.IPricipal, principal) as a muffler, but got fed up with it. So, upvote.

Yeah, there shouldn't be a need to do that at all though. It seems to not throw the same type errors in VS Code, but it's definitely a bug with jsii due to how it transforms TypeScript code to other languages. Python is weird in general IMHO.

[gshpychka]

Possible duplicate of #2927 / #4541

[iliapolo]

This issue has become hard to follow because it surfaces many different issues. I am closing it out as all issues have dedicated issues for them. See:

• (Python): @jsii.python.classproperty not working as intended with type checking #3633
• (@aws_cdk): Python: incompatible interface implementations due to differing function argument names #4541
• MyPy throwing type errors when it shouldn't #3931

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.