Pyppeteer: Browser closed unexpectedly in AWS Lambda

Sudhakar

I'm running into this error in AWS Lambda. It appears that the devtools websocket is not up. Not sure how to fix it. Any ideas? Thanks for your time.

Exception originated from get_ws_endpoint() due to websocket response timeout https://github.com/pyppeteer/pyppeteer/blob/ad3a0a7da221a04425cbf0cc92e50e93883b077b/pyppeteer/launcher.py#L225

Lambda code:

import os
import json
import asyncio
import logging
import boto3
import pyppeteer
from pyppeteer import launch

logger = logging.getLogger()
logger.setLevel(logging.INFO)

pyppeteer.DEBUG = True  # print suppressed errors as error log

def lambda_handler(event, context):
    asyncio.get_event_loop().run_until_complete(main())

async def main():
    browser = await launch({
        'headless': True,
        'args': [
            '--no-sandbox'
            ]
    })
    page = await browser.newPage()
    await page.goto('http://example.com')
    await page.screenshot({'path': '/tmp/example.png'})
    await browser.close()
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Exception:

    Response:
{
  "errorMessage": "Browser closed unexpectedly:\n",
  "errorType": "BrowserError",
  "stackTrace": [
    "  File \"/var/task/lambda_handler.py\", line 23, in lambda_handler\n    asyncio.get_event_loop().run_until_complete(main())\n",
    "  File \"/var/lang/lib/python3.8/asyncio/base_events.py\", line 616, in run_until_complete\n    return future.result()\n",
    "  File \"/var/task/lambda_handler.py\", line 72, in main\n    browser = await launch({\n",
    "  File \"/opt/python/pyppeteer/launcher.py\", line 307, in launch\n    return await Launcher(options, **kwargs).launch()\n",
    "  File \"/opt/python/pyppeteer/launcher.py\", line 168, in launch\n    self.browserWSEndpoint = get_ws_endpoint(self.url)\n",
    "  File \"/opt/python/pyppeteer/launcher.py\", line 227, in get_ws_endpoint\n    raise BrowserError('Browser closed unexpectedly:\\n')\n"
  ]
}

Request ID:
"06be0620-8b5c-4600-a76e-bc785210244e"

Function Logs:
START RequestId: 06be0620-8b5c-4600-a76e-bc785210244e Version: $LATEST
---- files in /tmp ----
[W:pyppeteer.chromium_downloader] start chromium download.
Download may take a few minutes.

  0%|          | 0/108773488 [00:00<?, ?it/s]
 11%|█▏        | 12267520/108773488 [00:00<00:00, 122665958.31it/s]
 27%|██▋       | 29470720/108773488 [00:00<00:00, 134220418.14it/s]
 42%|████▏     | 46172160/108773488 [00:00<00:00, 142570388.86it/s]
 58%|█████▊    | 62607360/108773488 [00:00<00:00, 148471487.93it/s]
 73%|███████▎  | 79626240/108773488 [00:00<00:00, 154371569.93it/s]
 88%|████████▊ | 95754240/108773488 [00:00<00:00, 156353972.12it/s]
100%|██████████| 108773488/108773488 [00:00<00:00, 161750092.47it/s]
[W:pyppeteer.chromium_downloader] 
chromium download done.
[W:pyppeteer.chromium_downloader] chromium extracted to: /tmp/local-chromium/588429
-----
/tmp/local-chromium/588429/chrome-linux/chrome
[ERROR] BrowserError: Browser closed unexpectedly:

Traceback (most recent call last):
  File "/var/task/lambda_handler.py", line 23, in lambda_handler
    asyncio.get_event_loop().run_until_complete(main())
  File "/var/lang/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/var/task/lambda_handler.py", line 72, in main
    browser = await launch({
  File "/opt/python/pyppeteer/launcher.py", line 307, in launch
    return await Launcher(options, **kwargs).launch()
  File "/opt/python/pyppeteer/launcher.py", line 168, in launch
    self.browserWSEndpoint = get_ws_endpoint(self.url)
  File "/opt/python/pyppeteer/launcher.py", line 227, in get_ws_endpoint
    raise BrowserError('Browser closed unexpectedly:\n')END RequestId: 06be0620-8b5c-4600-a76e-bc785210244e
REPORT RequestId: 06be0620-8b5c-4600-a76e-bc785210244e  Duration: 33370.61 ms   Billed Duration: 33400 ms   Memory Size: 3008 MB    Max Memory Used: 481 MB Init Duration: 445.58 ms    
Sudhakar

Answering my own question.

Finally I was able to run Pyppeteer(v0.2.2) with Python 3.6 and 3.7 (not 3.8) after I bundled chromium binary in a lambda layer.

So in summary, it appears to work only when its configured to run with user provided chromium executable path and not with automatically downloaded chrome. Probably some race condition or something.

Got Chromium from https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-41/stable-headless-chromium-amazonlinux-2017-03.zip

browser = await launch(
        headless=True, 
        executablePath='/opt/python/headless-chromium',
        args=[
            '--no-sandbox',
            '--single-process',
            '--disable-dev-shm-usage',
            '--disable-gpu',
            '--no-zygote'
        ])

Issue posted on repo https://github.com/pyppeteer/pyppeteer/issues/108

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Connect to a browser with Pyppeteer

Is it possible to simulate a browser with AWS lambda?

RuntimeError: Event loop is closed. Pyppeteer

Pull Request closed unexpectedly

KDevelop crashed (closed unexpectedly)

unexpectedly closed network connection

Getting "The inner stream position has changed unexpectedly" in AWS Lambda

Python: keep open browser in pyppeteer and create CDPSession

"Connection unexpectedly closed" error while sending email using SES from AWS

MSDeploy "The underlying connection was closed: The connection closed unexpectedly"

Cleaning up after AWS Lambda execution context is closed with Python

AWS lambda nodejs runtime: io: read/write on closed pipe

AWS lambda with python asyncio. Event loop closed problem?

Xamarin Android application closed unexpectedly

Unexpectedly closed channel in sync::mpsc

psql server closed the connection unexpectedly

server starts to closed the connection unexpectedly

How to fetch a url asynchronously with pyppeteer(One browser many tabs)

C# AWS Lambda API Gateway from browser - parameters not working

AWS Lambda & API Gateway - 500 from browser but not from curl

Nameko/RabbitMQ: OSError: Server unexpectedly closed connection

Postgres docker "server closed the connection unexpectedly"

IONIC : A utility CLI has unexpectedly closed

Django sending email: Connection unexpectedly closed

Putty: 'Server unexpectedly closed network connection'

postgres, server closed the connection unexpectedly via VBS

SMTPServerDisconnected: Connection unexpectedly closed: timed out

smtplib.SMTPServerDisconnected: Connection unexpectedly closed

CloseableHttpAsyncClient terminates with ConnectionClosedException: Connection closed unexpectedly