Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document about the ways to write unit tests for Bolt apps #380

Open
offbyone opened this issue Jun 15, 2021 · 18 comments
Open

Document about the ways to write unit tests for Bolt apps #380

offbyone opened this issue Jun 15, 2021 · 18 comments
Labels
area:async area:sync docs Improvements or additions to documentation enhancement New feature or request
Milestone

Comments

@offbyone
Copy link

I am looking to write unit tests for my Bolt API, which consists of an app with several handlers for events / messages. The handlers are both decorated using the app.event decorator, and make use of the app object to access things like the db connection that has been put on it. For example:

# in main.py
app = App(
    token=APP_TOKEN,
    signing_secret=SIGNING_SECRET,
    process_before_response=True,
    token_verification_enabled=token_verification_enabled,
)
app.db = db

# in api.py:
from .main import app

@app.command("/slashcommand")
def slash_command(ack, respond):
    ack()
    results = app.db.do_query()
    respond(...)

The thing is, I cannot find any framework pointers, or documentation, on how I would write reasonable unit tests for this. Presumably ones where the kwargs like ack and respond are replaced by test doubles. How do I do this?

The page URLs

Requirements

Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

@seratch seratch added area:async area:sync enhancement New feature or request docs Improvements or additions to documentation labels Jun 15, 2021
@seratch seratch added this to the 2.0.0 milestone Jun 15, 2021
@seratch seratch changed the title Please document how to write unit tests for Bolt APIs. Document about the ways to write unit tests for Bolt apps Jun 15, 2021
@seratch
Copy link
Member

seratch commented Jun 15, 2021

Hi @offbyone, thanks for taking the time to write in!

As we have been receiving feedback about testing, it's on our radar. I cannot tell when we can release built-in test support but I myself will be working on it in the latter half of this year.

At this moment, the only thing I can suggest is to do something similar with the tests this project has. You can find so many tests under this package: https://github.com/slackapi/bolt-python/tree/v1.6.1/tests/scenario_tests

The key points are:

  • Have your own mock server like this, which responds to Web API calls / response_url calls from test code. You don't need to use the standard HTTP server module for it.
  • App instances in tests need to have client argument in its constructor. The client's base_url needs to point the mock server's URL this way
  • Prepare request payload example data like this

In addition to above,

I cannot find any framework pointers

You can use any testing frameworks (my recommendation is pytest). The forthcoming testing support won't require your tests to rely on a specific test framework. We may provde some additional utility for pytest etc. for convenience, though.

Presumably ones where the kwargs like ack and respond are replaced by test doubles. How do I do this?

For respond, you can have a local mock HTTP server for handling the request. Regarding the ack call, it will be eventually converted to either an HTTP response or WebSocket message (Socket Mode) but before that, you can verify the value by checking response: Response returned by App#dispatch method as this test does.

I know that this approach is not so easy but it should be reasonable enough. Please consider going with it for now.

I hope this was helpful to you. If everything is clear for now, would you mind closing this issue? For the testing support task, I will come up with a bit more detailed issue soon. Also, for better documentation, we will add the section for testing once we release built-in testing support.

@offbyone
Copy link
Author

Yeah, it ... helps. I guess I'd consider, if I were you, leaving this issue open to track that there is a gap, and resolving it when that documentation exists. If you don't track issues that way, please feel free to close it, but as far as I'm concerned this is still a bug until I can go to your documentation site and see instructions on how to test my bot :D

@seratch
Copy link
Member

seratch commented Jun 16, 2021

if I were you, leaving this issue open to track that there is a gap, and resolving it when that documentation exists

Yeah, I understand this way. It's also fine to keep this open for now. I have a link to this in the new issue that organizes the tasks/discussions around testing support. Thanks for the nice suggestion.

@offbyone
Copy link
Author

I have a further question: Since this kind of testing requires constructing the app object differently, but the object is created at the module level, does someone there have a working pattern for how to do this in the bot code itself? Your example tests all create the app in the tests, but in a real Bolt application by the time you've imported the code, the App already exists and has been configured. So it's really tricky trying to sort out the right method here.

@carlos-alberto
Copy link

Hey @seratch is there any update on when this will be released?

@seratch
Copy link
Member

seratch commented Feb 8, 2022

@carlos-alberto I did a quick prototyping late last year. After that, we haven't had good progress due to other priorities. But we're aiming to provide a better solution sometime this year!

@I-Dont-Remember
Copy link

If it helps those who stumble on this in the future, this is an open-source Bolt app with a moderately sized test suite as an example to dig through.

We've taken the approach of keeping our core functionality in a separate module that doesn't depend on slack_bolt, letting us easily run tests without dealing with bolt trying to start a server or anything like that.

Then in app.py you can just import the module and use the Slack @decorated functions essentially as wrappers over your unit-testable business logic.

This only helps for unit tests, unfortunately haven't delved into integration tests, so unfortunately no review/feedback on seratch's suggestion.

@jimenezj8
Copy link

Any progress on this since the last update?

@seratch
Copy link
Member

seratch commented Jul 18, 2023

Hey @jimenezj8, this is still on my radar and actually I did quick prototyping last year. However, I cannot tell when we can deliver a stable solution due to our bandwidth and priorities. We will update you when we come up with something in the future.

@jimenezj8
Copy link

Thanks for the update @seratch 🙂 will keep my eyes peeled for another

@fr12k
Copy link

fr12k commented Sep 14, 2023

Hi i was looking into how to write unit tests for our slack bot and found this wonderful library.
https://github.com/gabrielfalcao/HTTPretty
That make's it easy to mock http requests.

Just be aware that the 1.1 version is broken if you want to use it use 1.0.5.
gabrielfalcao/HTTPretty#425

@ChadDa3mon
Copy link

Came across this as well and figured I'd try to contribute a bit. Here's some modified code from my current app, hopefully it can help someone else. Normally I'd have all of the payload stuff in my conftest.py file to keep things clean, but for the purposes of this I figured I'd put it all in one place.

import pytest
from unittest.mock import AsyncMock
from my.production.code import easter_egg

# This is the shortcut we expect to receive from Slack (well, a heavily truncated one)
mock_shortcut = {
        "message": {
            "text": "Hello, World!",
            "ts": "1234567890.123456"
        },
        "user": {
            "id": "U123456",
            "name": "John Doe"
        },
        "channel": {
            "name": "general"
        },
        "trigger_id": "trigger_12345"
    }

# Upon receiving this shortcut, we will send a 'view' to our user
# This is what we expect it to look like
expected_modal = {
    "type": "modal",
    "callback_id": "hello-modal",
    "title": {
        "type": "plain_text",
        "text": "Greetings!"
    },
    "submit": {
        "type": "plain_text",
        "text": "Good Bye"
    },
    "blocks":[
        {
          "type": "section",
          "text": {
            "type": "plain_text",
            "text": "Frosty the Snowman",
            "emoji": True
          }
        },
        {
          "type": "video",
          "title": {
            "type": "plain_text",
            "text": "Frosty the Snowman",
            "emoji": True
          },
          "title_url": "https://www.youtube.com/watch?v=RRxQQxiM7AA",
          "description": {
            "type": "plain_text",
            "text": "Frosty the Snowman",
            "emoji": True
          },
          "video_url": "https://www.youtube.com/embed/vjscH2WBWjw?feature=oembed&autoplay=1",
          "alt_text": "Frosty the Snowman",
          "thumbnail_url": "https://i.ytimg.com/vi/bSzBBK8gC6c/hqdefault.jpg",
          "author_name": "",
          "provider_name": "YouTube",
          "provider_icon_url": "https://a.slack-edge.com/80588/img/unfurl_icons/youtube.png"
        }
    ]
}

@pytest.mark.asyncio
async def test_easter_egg():
    ack = AsyncMock()
    client = AsyncMock()
    client.views_open = AsyncMock()

    # Test my actual production function 'easter_egg'
    # This should ack the shortcut and then open a view to the user
    await easter_egg(ack, mock_shortcut, client)
    ack.assert_awaited_once # Make sure it was acknowledged
    client.views_open.assert_awaited_once_with( # Make sure the view we get back is correct
        trigger_id=mock_shortcut["trigger_id"],
        view=expected_modal
    )

@leon-adarcewicz
Copy link

Any progress on this since the last update?

@zimeg
Copy link
Member

zimeg commented Jul 9, 2024

@leon-adarcewicz the bolt-python-jira-functions sample app can be good reference with a few of the unit tests for listeners - similar to what @ChadDa3mon shared above! 🧪

The tests for Workflow Buddy from @I-Dont-Remember are also solid at testing the overall listener logic while mocking HTTP requests is another great option from @fr12k 🙏

I'm not aware of updates to built-in test helpers for Bolt Python, but these all seem like good testing approaches to me. I hope this is helpful for now, and in some neartime I'll explore this a bit more to include on slack.dev/bolt-python 📚

@leon-adarcewicz
Copy link

Thank you @zimeg!

@dsoprea
Copy link

dsoprea commented Jul 11, 2024

@seratch Any feedback on the current state of unit-testing support?

@seratch
Copy link
Member

seratch commented Jul 11, 2024

No updates from me lately. To align with our platform roadmap and priorities, I am not actively working on the prototype right now.

@NathanSmeltzer
Copy link

This may help with testing, but I haven't tried it yet:
https://github.com/ygalblum/slack-server-mock

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:async area:sync docs Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests