[bug] getStoryContext() does not include all the story tags as defined in the index.json file

[rgylesbedford]

Describe the bug

I am attempting to use the tags to opt-out of specific tests within the test runner.
The tags array on the context object returned from getStoryContext() does not match the tags as defined in the index.json file.

To Reproduce

1. Storybook is built using: storybook build --test
2. Test Runner is run with: test-storybook --junit --failOnConsole --ci --url http://127.0.0.1:6006
3. Using the following simplified ./storybook/test-runner.ts file; the tags array on the storyContext object is always ["story"].

import type { TestRunnerConfig } from "@storybook/test-runner";
import { getStoryContext, waitForPageReady } from "@storybook/test-runner";
import { injectAxe, checkA11y } from "axe-playwright";

// See https://storybook.js.org/docs/writing-tests/test-runner#test-hook-api
const config: TestRunnerConfig = {
  tags: {
    skip: ["skip-test"],
  },
  async preVisit(page) {
    await injectAxe(page);
  },
  async postVisit(page, context) {
    // See https://storybook.js.org/docs/writing-tests/test-runner#helpers
    const storyContext = await getStoryContext(page, context);

    const skipA11y = storyContext.tags.includes("skip-a11y");
    const skipVisual = storyContext.tags.includes("skip-visual");

    // See https://storybook.js.org/docs/writing-tests/storyshots-migration-guide#run-image-snapshot-tests-with-the-test-runner
    if(!skipVisual) {
      // removed for simplicity
    }
    
    // See https://storybook.js.org/docs/writing-tests/accessibility-testing#disable-a11y-tests-with-the-test-runner
    if(!skipA11y) {
      await checkA11y(page, '#storybook-root', {
        detailedReport: true,
        detailedReportOptions: {
          html: true,
        },
      });
    }
  }
};

export default config;

Expected behavior

The tags array on storyContext object matches the tags listed in the index.json file for the given story.

System

Storybook Environment Info:

  System:
    OS: Windows 11 10.0.22621
    CPU: (16) x64 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
  Binaries:
    Node: 18.20.1 - C:\Program Files\nodejs\node.EXE
    npm: 10.5.2 - C:\Program Files\nodejs\npm.CMD <----- active
  Browsers:
    Edge: Chromium (123.0.2420.97)
  npmPackages:
    @storybook/addon-a11y: ^8.0.9 => 8.0.9
    @storybook/addon-actions: ^8.0.9 => 8.0.9
    @storybook/addon-essentials: ^8.0.9 => 8.0.9
    @storybook/addon-interactions: ^8.0.9 => 8.0.9
    @storybook/addon-links: ^8.0.9 => 8.0.9
    @storybook/addon-storysource: ^8.0.9 => 8.0.9
    @storybook/blocks: ^8.0.9 => 8.0.9
    @storybook/preview-api: ^8.0.9 => 8.0.9
    @storybook/react: ^8.0.9 => 8.0.9
    @storybook/react-vite: ^8.0.9 => 8.0.9
    @storybook/test: ^8.0.9 => 8.0.9
    @storybook/test-runner: ^0.17.0 => 0.17.0
    eslint-plugin-storybook: ^0.8.0 => 0.8.0
    storybook: ^8.0.9 => 8.0.9
    storybook-dark-mode: ^4.0.1 => 4.0.1

Additional context

Example snippet from the index.json file:

{
  "v": 4,
  "entries": {
    "examplecomponent--default": {
      "type": "story",
      "id": "examplecomponent--default",
      "name": "Default",
      "title": "ExampleComponent",
      "importPath": "./src/components/ExampleComponent/ExampleComponent.stories.tsx",
      "tags": ["skip-test", "story"]
    },
    "othercomponent--default": {
      "type": "story",
      "id": "othercomponent--default",
      "name": "Default",
      "title": "OtherComponent",
      "importPath": "./src/components/OtherComponent/OtherComponent.stories.tsx",
      "tags": ["skip-a11y", "story"]
    }
  }
}

[shilman]

Tag handling changed dramatically in 8.1.0, so please give it another try int he latest version and let me know what you find.

[shilman]

Also it's possible some changes are needed in the test runner as well @yannbf #453

[rgylesbedford]

Tag handling changed dramatically in 8.1.0, so please give it another try in the latest version and let me know what you find.

Thanks. I got to testing the latest storybook this week and can confirm this is now working for me.

Storybook Environment Info:

  System:
    OS: Windows 11 10.0.22621
    CPU: (16) x64 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
  Binaries:
    Node: 20.13.0 - C:\Program Files\nodejs\node.EXE
    npm: 10.5.2 - C:\Program Files\nodejs\npm.CMD <----- active
  Browsers:
    Edge: Chromium (125.0.2535.79)
  npmPackages:
    @storybook/addon-a11y: ^8.1.5 => 8.1.5
    @storybook/addon-actions: ^8.1.5 => 8.1.5
    @storybook/addon-essentials: ^8.1.5 => 8.1.5
    @storybook/addon-interactions: ^8.1.5 => 8.1.5
    @storybook/addon-links: ^8.1.5 => 8.1.5
    @storybook/addon-storysource: ^8.1.5 => 8.1.5
    @storybook/blocks: ^8.1.5 => 8.1.5
    @storybook/preview-api: ^8.1.5 => 8.1.5
    @storybook/react: ^8.1.5 => 8.1.5
    @storybook/react-vite: ^8.1.5 => 8.1.5
    @storybook/test: ^8.1.5 => 8.1.5
    @storybook/test-runner: ^0.18.2 => 0.18.2
    eslint-plugin-storybook: ^0.8.0 => 0.8.0
    storybook: ^8.1.5 => 8.1.5
    storybook-dark-mode: ^4.0.1 => 4.0.1