Update puppeteer to version 23.4.0

[timvandermeij]

In #18771 we found that updating Puppeteer from version 23.3.0 to 23.4.0 caused new test failures. This is also reproducible locally. The logs from the bots can be found at http://54.193.163.58:8877/f9b8d1c303f789c/output.txt (Windows) and http://54.241.84.105:8877/902fcf449da1c69/output.txt (Linux). Version 23.3.1 doesn't have this issue locally, so we need to investigate what changed between 23.3.1 and 23.4.0 that causes some tests to fail and/or harden the tests against (intermittent) failure.

[timvandermeij]

I have compared the Puppeteer 23.3.1 release with the Puppeteer 23.4.0 release and found the following commits: puppeteer/puppeteer@puppeteer-v23.3.1...puppeteer-v23.4.0. There isn't really much interesting in there other than the Chrome update, and in the logs above I noticed that the failures are all in Chrome. However, the problems in Chrome might point to pre-existing issues in the tests themselves that only surface now.

[jamesgpearce]

Unrelated to pdf.js but I am also getting unusual errors from 23.4 relative to 23.3. Basic things like not being able to connect to the server for the first few tests.

[timvandermeij]

The issue exists in Puppeteer 23.5.0 too. The following logs contain the failures for me locally when running the integration tests against that version:

Puppeteer 23.5.0 integration test logs

Failures:
1) FreeText Editor Freetext must stay focused after having been moved must keep the focus
  Message:
    In chrome: Expected 0 to equal 1.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/freetext_editor_spec.mjs:2326:14
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/freetext_editor_spec.mjs:2250:7)

2) Highlight Editor Highlight with the keyboard must check that some text has been highlighted
  Message:
    ProtocolError: Waiting for selector `#pdfjs_internal_editor_0` failed: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

3) Highlight Editor Highlight and caret browsing must check that selection is correctly highlighted on arrow down key pressed
  Message:
    ProtocolError: Waiting for selector `#pdfjs_internal_editor_0` failed: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

4) Highlight Editor Use a toolbar overlapping an other highlight must check that the toolbar is usable
  Message:
    ProtocolError: Waiting for selector `#pdfjs_internal_editor_0` failed: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

5) Ink Editor Basic operations must draw, undo/redo and check that the editor don't move
  Message:
    In chrome: Expected 420 to equal 121.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:133:14
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:103:7)
  Message:
    In chrome: Expected 331 to equal 32.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:137:14
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/ink_editor_spec.mjs:103:7)

6) Interaction in issue15092.pdf must check that a value is correctly updated on a field and its siblings
  Message:
    ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

7) Interaction in bug1802888.pdf must check field value is treated by default as a number
  Message:
    In chrome: Expected '479' to equal '579'.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/scripting_spec.mjs:1596:58
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/scripting_spec.mjs:1580:7)

8) Interaction Textfield with a zip code starting with 0 must check the zip code is correctly formatted
  Message:
    ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:270:36)
        at IsolatedWorld.evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:97:30)
        at CdpJSHandle.evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:139:37)
        at awaitPromise (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/test_utils.mjs:129:18)

9) Stamp Editor Basic operations must load a SVG, delete it and undo
  Message:
    ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
  Stack:
        at <instance_members_initializer> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:89:14)
        at new Callback (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:93:16)
        at CallbackRegistry.create (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
        at Connection._rawSend (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:86:26)
        at CdpCDPSession.send (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
        at #evaluate (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:355:50)
        at ExecutionContext.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:322:36)
        at IsolatedWorld.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/IsolatedWorld.js:86:30)
        at CdpJSHandle.evaluateHandle (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/api/JSHandle.js:147:37)
        at WaitTask.rerun (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:100:47)

10) Text layer Text selection using mouse doesn't jump when hovering on an empty area in a single page
  Message:
    In chrome:
        Expected '' to roughly match 'code sequences, records
        them, and compiles them to fast native code. We call suc'.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:138:18
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:116:11

In contrast, running the integration test locally with the currently pinned Puppeteer 23.3.1 version gives the following much cleaners logs (the only failure here is tracked in #18774):

Puppeteer 23.3.1 integration test logs

Failures:
1) Text layer Text selection using mouse doesn't jump when hovering on an empty area in a single page
  Message:
    In chrome:
        Expected '' to roughly match 'code sequences, records
        them, and compiles them to fast native code. We call suc'.
  Stack:
        at <Jasmine>
        at file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:138:18
        at async Promise.all (index 1)
        at async UserContext.<anonymous> (file:///home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/test/integration/text_layer_spec.mjs:116:11)