{"id":14618,"date":"2023-01-12T20:42:29","date_gmt":"2023-01-12T20:42:29","guid":{"rendered":"https:\/\/cheesecakelabs.com\/blog\/selenium-web-scraping-aws-lambda\/"},"modified":"2026-08-15T07:05:52","modified_gmt":"2026-08-15T07:05:52","slug":"selenium-web-scraping-aws-lambda","status":"publish","type":"post","link":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/","title":{"rendered":"Como usar Selenium para web scraping no AWS Lambda"},"content":{"rendered":"\n<p>Web scraping pode economizar v\u00e1rias horas em compara\u00e7\u00e3o com a coleta manual de dados em sites, e o AWS Lambda \u00e9 uma boa forma de configurar scripts para rodar sob demanda.<\/p>\n\n\n\n<p>Neste artigo, voc\u00ea vai ver o que precisa saber sobre web scraping e AWS Lambda, com o que cada um tem de bom e de ruim. Tamb\u00e9m vamos analisar o c\u00f3digo de um projeto de exemplo que combina essas duas tecnologias, que se encaixam muito bem.<\/p>\n\n\n\n<p>Se voc\u00ea n\u00e3o tem um bom jeito de <strong>extrair informa\u00e7\u00e3o de um site<\/strong>, data scraping \u00e9 o caminho. Al\u00e9m disso, \u00e9 muito bom ver um navegador abrindo, clicando em bot\u00f5es e preenchendo formul\u00e1rios sozinho.<\/p>\n\n\n\n<p>O Selenium foi o padr\u00e3o por 15 anos. O Playwright surgiu em 2020 e vem ganhando espa\u00e7o de forma consistente. Hoje a pergunta n\u00e3o \u00e9 mais \u201cdevo aprender Selenium?\u201d, e sim \u201cquando Selenium ainda faz sentido?\u201d.<\/p>\n\n\n\n<p>Vamos falar sobre essas tecnologias e passar por um exemplo em Python que resolve esse mist\u00e9rio. Ou, se voc\u00ea n\u00e3o est\u00e1 a fim de ler defini\u00e7\u00f5es e quer ir direto ao ponto, pule para a se\u00e7\u00e3o \u201cO c\u00f3digo do projeto\u201d.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"Data-Scraping\">O que \u00e9 data scraping e quando ele se justifica<\/h2>\n\n\n\n<p>Data scraping \u00e9 o processo de extrair informa\u00e7\u00e3o gerada por outro programa. Web scraping \u00e9 o caso espec\u00edfico em que essa informa\u00e7\u00e3o est\u00e1 em um site.<\/p>\n\n\n\n<p>Trate isso como \u00faltimo recurso. APIs existem justamente para dar acesso estruturado e confi\u00e1vel aos dados que voc\u00ea precisa. <a href=\"https:\/\/cheesecakelabs.com\/blog\/api-design-think-first-code-later\/\" target=\"_blank\" rel=\"noreferrer noopener\">Consumir uma API<\/a> \u00e9 mais r\u00e1pido, mais est\u00e1vel e n\u00e3o carrega nenhuma das fragilidades legais ou t\u00e9cnicas que o scraping traz. Se existe uma API, use a API.<\/p>\n\n\n\n<p>O scraping se justifica quando n\u00e3o existe API, os dados s\u00e3o de acesso p\u00fablico e a estrutura do site \u00e9 est\u00e1vel. O m\u00e9todo que voc\u00ea escolhe depende do que vai ser coletado.<\/p>\n\n\n\n<p><strong>Quando voc\u00ea n\u00e3o precisa de um navegador:<\/strong> se o dado que voc\u00ea quer j\u00e1 vem na resposta HTML inicial, uma abordagem HTTP leve \u00e9 imbat\u00edvel. Sem o custo de subir um navegador, sem execu\u00e7\u00e3o de JavaScript, sem ChromeDriver para gerenciar.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\"><span class=\"hljs-keyword\">from<\/span> bs4 <span class=\"hljs-keyword\">import<\/span> BeautifulSoup\n<span class=\"hljs-keyword\">from<\/span> urllib.request <span class=\"hljs-keyword\">import<\/span> urlopen\n\n<span class=\"hljs-keyword\">with<\/span> urlopen(<span class=\"hljs-string\">'https:\/\/en.wikipedia.org\/wiki\/Main_Page'<\/span>) <span class=\"hljs-keyword\">as<\/span> response:\n    soup = BeautifulSoup(response, <span class=\"hljs-string\">'html.parser'<\/span>)\n    <span class=\"hljs-keyword\">for<\/span> anchor <span class=\"hljs-keyword\">in<\/span> soup.find_all(<span class=\"hljs-string\">'a'<\/span>):\n        print(anchor.get(<span class=\"hljs-string\">'href'<\/span>, <span class=\"hljs-string\">'\/'<\/span>))<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>Essa abordagem, <code><strong>requests<\/strong><\/code> ou <code><strong>urllib<\/strong><\/code> mais<strong> <code>BeautifulSoup<\/code><\/strong> para o parsing do HTML, cobre a maioria dos casos simples de scraping e n\u00e3o exige infraestrutura de cloud al\u00e9m de uma fun\u00e7\u00e3o b\u00e1sica.<\/p>\n\n\n\n<p><strong>Quando voc\u00ea precisa de um navegador:<\/strong> p\u00e1ginas renderizadas por JavaScript, single-page applications, conte\u00fado din\u00e2mico carregado depois do load, fluxos de login, intera\u00e7\u00e3o com formul\u00e1rios. Tudo isso exige automa\u00e7\u00e3o de navegador. \u00c9 aqui que Selenium e Playwright entram, e \u00e9 aqui que o Lambda vira o alvo certo para o deploy.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"List-of-some-web-scraping-techniques:\"><strong>T\u00e9cnicas de web scraping: uma refer\u00eancia r\u00e1pida<\/strong><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Copiar e colar<\/strong>: manual. Todo desenvolvedor j\u00e1 fez isso. N\u00e3o vale automatizar para uma tarefa pontual.<\/li>\n\n\n\n<li><strong>Casamento de padr\u00f5es em texto<\/strong>: usar ferramentas como <code>curl<\/code> e <code>grep<\/code> para extrair padr\u00f5es do HTML bruto. R\u00e1pido para alvos simples e est\u00e1veis, com markup previs\u00edvel.<\/li>\n\n\n\n<li><strong>Parsing de HTML<\/strong>: transformar a resposta em uma \u00e1rvore de elementos naveg\u00e1vel. O <code>BeautifulSoup<\/code> \u00e9 a biblioteca padr\u00e3o em Python. N\u00e3o precisa de navegador. Serve para conte\u00fado est\u00e1tico.<\/li>\n\n\n\n<li><strong>Parsing do DOM:<\/strong> um navegador controlado por software de automa\u00e7\u00e3o interage com a p\u00e1gina como um usu\u00e1rio real: abre p\u00e1ginas, clica em bot\u00f5es, preenche formul\u00e1rios, espera o JavaScript executar e ent\u00e3o extrai os dados do DOM renderizado. \u00c9 o territ\u00f3rio do Selenium e do Playwright. Tamb\u00e9m \u00e9 a abordagem que mais consome recursos e a que exige mais infraestrutura para rodar de forma confi\u00e1vel na cloud.<\/li>\n<\/ul>\n\n\n\n<p>Um pacote conhecido para isso \u00e9 o <a href=\"https:\/\/www.selenium.dev\/\" target=\"_blank\" rel=\"noreferrer noopener\">Selenium<\/a>. Essa t\u00e9cnica exige um driver para se comunicar com o navegador instalado (por exemplo, <a href=\"https:\/\/chromedriver.chromium.org\/downloads\" target=\"_blank\" rel=\"noreferrer noopener\">ChromeDriver<\/a>, <a href=\"https:\/\/github.com\/mozilla\/geckodriver\/releases\" target=\"_blank\" rel=\"noreferrer noopener\">GeckoDriver<\/a>).<\/p>\n\n\n\n<p>Esse driver exp\u00f5e uma API para que o pacote Selenium consiga controlar o navegador. A vers\u00e3o do driver precisa ser compat\u00edvel com a vers\u00e3o do navegador instalado.<\/p>\n\n\n\n<p>Veja um exemplo de web scraper que abre um navegador, pega o elemento de cabe\u00e7alho da p\u00e1gina e imprime o conte\u00fado dele:<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:100%\"><pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">from<\/span> selenium <span class=\"hljs-keyword\">import<\/span> webdriver\n<span class=\"hljs-keyword\">from<\/span> selenium.webdriver.common.by <span class=\"hljs-keyword\">import<\/span> By\n\ndriver = webdriver.Chrome() <span class=\"hljs-comment\"># Open browser<\/span>\ndriver.get(<span class=\"hljs-string\">\"http:\/\/example.com\"<\/span>) <span class=\"hljs-comment\"># Access page<\/span>\nheader_element = driver.find_element(By.CSS_SELECTOR, <span class=\"hljs-string\">'h1'<\/span>) <span class=\"hljs-comment\"># Find H1 element<\/span>\nprint(header_element.text) <span class=\"hljs-comment\"># Print found element text content<\/span>\ndriver.quit() <span class=\"hljs-comment\"># Close browser<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre><\/div>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"How-to-use-the-Web-Scraper-script?\"><strong>Por que usar AWS Lambda para web scraping<\/strong><\/h2>\n\n\n\n<p>Neste artigo, vamos colocar o script do web scraper na <strong>cloud<\/strong>. Computa\u00e7\u00e3o em nuvem disponibiliza servi\u00e7os como hospedagem e armazenamento pela internet.<\/p>\n\n\n\n<p>Alguns provedores de cloud conhecidos s\u00e3o <a href=\"https:\/\/aws.amazon.com\/?nc2=h_lg\" target=\"_blank\" rel=\"noreferrer noopener\">Amazon AWS<\/a>, <a href=\"https:\/\/cloud.google.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Google Cloud<\/a> e <a href=\"https:\/\/azure.microsoft.com\/en-us\/\" target=\"_blank\" rel=\"noreferrer noopener\">Microsoft Azure<\/a>. Usar uma solu\u00e7\u00e3o em nuvem traz v\u00e1rias vantagens, por exemplo:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Voc\u00ea n\u00e3o precisa configurar o seu servidor<\/li>\n\n\n\n<li>H\u00e1 muitos engenheiros trabalhando para garantir que esses servi\u00e7os n\u00e3o tenham falhas de seguran\u00e7a<\/li>\n\n\n\n<li>Normalmente \u00e9 muito f\u00e1cil escalar a capacidade do servi\u00e7o de cloud que voc\u00ea est\u00e1 usando. Em muitos deles, d\u00e1 para aumentar mem\u00f3ria e poder de processamento clicando em alguns bot\u00f5es<\/li>\n<\/ul>\n\n\n\n<p>Alguns desses <a href=\"https:\/\/cheesecakelabs.com\/blog\/servicos-de-cloud-para-seu-projeto\/\" target=\"_blank\" rel=\"noreferrer noopener\">servi\u00e7os de cloud<\/a> s\u00e3o <strong>serverless<\/strong>, ou seja, rodam sob demanda e o provedor cuida da infraestrutura de servidor no lugar do cliente.&nbsp;<\/p>\n\n\n\n<p>Outros exigem o uso de <strong>containers<\/strong>, que s\u00e3o execu\u00e7\u00f5es de sistemas operacionais virtualizados a partir de imagens (templates). O <a href=\"https:\/\/www.docker.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Docker<\/a> \u00e9 a principal plataforma para trabalhar com containers.<\/p>\n\n\n\n<p>Computa\u00e7\u00e3o em nuvem \u00e9 \u00f3tima, mas n\u00e3o \u00e9 necess\u00e1ria em todo projeto. Rodar o script na sua m\u00e1quina pode ser suficiente.&nbsp;<\/p>\n\n\n\n<p>Cada um dos provedores citados tem v\u00e1rias formas de hospedar um script de web scraper. <a href=\"https:\/\/aws.amazon.com\/blogs\/architecture\/serverless-architecture-for-a-web-scraping-solution\/\" target=\"_blank\" rel=\"noreferrer noopener\">Este artigo do blog da AWS<\/a> descreve 3 op\u00e7\u00f5es.<\/p>\n\n\n\n<p><strong>Resumindo o artigo, as op\u00e7\u00f5es s\u00e3o:<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Criar uma m\u00e1quina virtual com o servi\u00e7o EC2. \u00c9 a op\u00e7\u00e3o mais primitiva: voc\u00ea precisa configurar a m\u00e1quina como faria com qualquer outra, e ela fica ligada 24 horas por dia. Tamb\u00e9m \u00e9 a solu\u00e7\u00e3o mais cara das 3 op\u00e7\u00f5es.<\/li>\n\n\n\n<li>Colocar o script em um container e us\u00e1-lo no servi\u00e7o AWS Fargate. O Fargate \u00e9 uma op\u00e7\u00e3o serverless, o que ajuda porque o web scraper s\u00f3 precisa rodar sob demanda. O Fargate tamb\u00e9m \u00e9 mais barato que o EC2.<\/li>\n\n\n\n<li>Usar o AWS Lambda, que tamb\u00e9m \u00e9 um servi\u00e7o serverless e aceita tanto c\u00f3digo puro quanto scripts em container. Tem mais limita\u00e7\u00f5es que o Fargate, mas resolve na maioria dos casos. \u00c9 o servi\u00e7o mais barato, e o seu script pode at\u00e9 caber no free tier.<\/li>\n<\/ol>\n\n\n\n<p>Aqui vamos usar o AWS Lambda. As principais limita\u00e7\u00f5es dele s\u00e3o o timeout, de 15 minutos, e o pacote de deploy, que n\u00e3o pode passar de 250 MB (mas aceita at\u00e9 10 GB usando containers).<\/p>\n\n\n\n<p>O script de exemplo \u00e9 bem simples e leva menos de um minuto para rodar. Mas, como j\u00e1 dissemos, o Selenium exige um navegador, e o bin\u00e1rio do Chrome tem cerca de 500 MB, o que nos obriga a usar a abordagem de container.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p><strong>Leia tamb\u00e9m: <\/strong><a href=\"https:\/\/cheesecakelabs.com\/blog\/boas-praticas-finops-aws-custos-cloud\/\" id=\"13020\" target=\"_blank\" rel=\"noreferrer noopener\">Boas pr\u00e1ticas de FinOps na AWS: como cortar e otimizar custos de cloud<\/a><\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"How-are-we-going-to-deploy-it-to-AWS-Lambda?\"><strong>Como fazer o deploy no AWS Lambda<\/strong><\/h2>\n\n\n\n<p>Existem v\u00e1rias formas de configurar uma fun\u00e7\u00e3o no AWS Lambda. Uma delas, bem simples, \u00e9 usar o <a href=\"https:\/\/serverless.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Serverless Framework<\/a>. Ele ajuda a desenvolver e fazer deploy de fun\u00e7\u00f5es Lambda com um \u00fanico arquivo YAML, que declara as fun\u00e7\u00f5es, a infraestrutura delas e os eventos que v\u00e3o dispar\u00e1-las.<\/p>\n\n\n\n<p>O Serverless Framework tamb\u00e9m permite fazer o deploy das fun\u00e7\u00f5es lambda com um \u00fanico comando, o que simplifica bastante o processo.<\/p>\n\n\n\n<p>O Serverless Framework ainda oferece um dashboard opcional, com uma interface para checar a sa\u00fade da fun\u00e7\u00e3o, disparar eventos manualmente e ver os logs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"The-codebase:\"><strong>O c\u00f3digo do projeto<\/strong><\/h2>\n\n\n\n<p>Nesta se\u00e7\u00e3o, vamos analisar alguns arquivos importantes de um projeto de demonstra\u00e7\u00e3o, que voc\u00ea pode ver <a href=\"https:\/\/github.com\/CheesecakeLabs\/selenium-serverless-example\" target=\"_blank\" rel=\"noreferrer noopener\">aqui<\/a>. <a href=\"https:\/\/github.com\/CheesecakeLabs\/selenium-serverless-example\/blob\/main\/serverless.yml\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>serverless.yml<\/strong><\/a><\/p>\n\n\n\n<p>\u00c9 neste arquivo que definimos a infraestrutura da aplica\u00e7\u00e3o lambda, as fun\u00e7\u00f5es lambda e os eventos que v\u00e3o dispar\u00e1-las.<\/p>\n\n\n\n<p>Na se\u00e7\u00e3o <strong>provider<\/strong> do arquivo, declaramos que vamos usar uma imagem docker chamada <em>img<\/em>. A se\u00e7\u00e3o <strong>functions<\/strong> \u00e9 onde definimos as fun\u00e7\u00f5es lambda e a configura\u00e7\u00e3o espec\u00edfica de cada uma, como vari\u00e1veis de ambiente e fun\u00e7\u00f5es handler.<\/p>\n\n\n\n<p>Repare que aqui adicionamos vari\u00e1veis de ambiente com o caminho do navegador e do driver. Declaramos que vamos usar a imagem definida antes, que o comando executado quando a fun\u00e7\u00e3o lambda for disparada \u00e9 o handler do arquivo <em>example.py <\/em>e que o evento que dispara tudo \u00e9 um <em>cronjob<\/em> agendado a cada 6 horas.<\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/CheesecakeLabs\/selenium-serverless-example\/blob\/main\/Dockerfile\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>Dockerfile<\/strong><\/a><\/p>\n\n\n\n<p>O Dockerfile \u00e9 o template onde configuramos a imagem do nosso container. Esse arquivo cria uma inst\u00e2ncia Linux capaz de rodar o web scraper. O template instala as depend\u00eancias do projeto (incluindo o Chrome e o driver do Chrome) e copia os arquivos necess\u00e1rios para a imagem.<\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/CheesecakeLabs\/selenium-serverless-example\/tree\/main\/src\/handlers\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>Example.py<\/strong><\/a><\/p>\n\n\n\n<p>Esse arquivo tem uma fun\u00e7\u00e3o que ser\u00e1 executada quando o evento for disparado. Ele \u00e9 bem parecido com o exemplo de Selenium que mostramos antes.<\/p>\n\n\n\n<p>A diferen\u00e7a principal \u00e9 que configuramos o navegador para n\u00e3o exibir interface (porque o lambda n\u00e3o tem display) e para usar um \u00fanico processo (porque o lambda tem apenas 1 CPU).<\/p>\n\n\n\n<p>Neste caso, a fun\u00e7\u00e3o handler retorna um dicion\u00e1rio com status code e body, para o caso de voc\u00ea querer trocar o evento de um cronjob para uma requisi\u00e7\u00e3o HTTP.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"Deploying\"><strong>Fazendo o deploy<\/strong><\/h3>\n\n\n\n<p>Para fazer o deploy da fun\u00e7\u00e3o lambda, basta rodar um \u00fanico comando no terminal.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1200\" height=\"275\" src=\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapper-aws-lambda-1200x275.jpg\" alt=\"Arquitetura de um web scraper com Selenium no AWS Lambda\" class=\"wp-image-11060\" srcset=\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapper-aws-lambda-1200x275.jpg 1200w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapper-aws-lambda-600x138.jpg 600w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapper-aws-lambda-768x176.jpg 768w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapper-aws-lambda-1536x352.jpg 1536w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapper-aws-lambda-760x174.jpg 760w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapper-aws-lambda.jpg 1858w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/figure>\n\n\n\n<p>Aqui o Serverless Framework emite um aviso explicando que o dashboard n\u00e3o suporta fun\u00e7\u00f5es que usam imagens de container. Ou seja, n\u00e3o vamos conseguir ver os logs da fun\u00e7\u00e3o lambda nem dispar\u00e1-la manualmente pelo dashboard.<\/p>\n\n\n\n<p>Mas ainda d\u00e1 para fazer isso pelo console da AWS. Abaixo, um print de um log de execu\u00e7\u00e3o bem-sucedida obtido no AWS Cloudwatch:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1200\" height=\"379\" src=\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapping-aws-1200x379.jpg\" alt=\"Configura\u00e7\u00e3o da fun\u00e7\u00e3o de web scraping no AWS Lambda\" class=\"wp-image-11062\" srcset=\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapping-aws-1200x379.jpg 1200w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapping-aws-600x190.jpg 600w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapping-aws-768x243.jpg 768w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapping-aws-1536x486.jpg 1536w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapping-aws-760x240.jpg 760w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/web-scrapping-aws.jpg 1999w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Selenium no Lambda: setup completo atualizado para 2026<\/h2>\n\n\n\n<p>A arquitetura continua a mesma do artigo original. As atualiza\u00e7\u00f5es est\u00e3o no runtime Python (3.12, a vers\u00e3o mais recente suportada pelo Lambda), na sintaxe da flag headless do Chrome (que mudou no Chrome 112) e no Serverless Framework v4.<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">scraper\/\n\u251c\u2500\u2500 Dockerfile\n\u251c\u2500\u2500 handler.py\n\u251c\u2500\u2500 requirements.txt\n\u2514\u2500\u2500 serverless.yml<\/code><\/span><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Dockerfile:<\/h3>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-3\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">FROM <span class=\"hljs-keyword\">public<\/span>.ecr.aws\/lambda\/python:<span class=\"hljs-number\">3.12<\/span>\n\n<span class=\"hljs-comment\"># Install Chromium and ChromeDriver<\/span>\nRUN dnf install -y chromium chromedriver\n\n<span class=\"hljs-comment\"># Install Python dependencies<\/span>\nCOPY requirements.txt .\nRUN pip install -r requirements.txt --target <span class=\"hljs-string\">\"${LAMBDA_TASK_ROOT}\"<\/span>\n\n<span class=\"hljs-comment\"># Copy handler<\/span>\nCOPY handler.py ${LAMBDA_TASK_ROOT}\n\nCMD &#91;<span class=\"hljs-string\">\"handler.handler\"<\/span>]<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-3\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>requirements.txt:<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">selenium==4.18.1<\/code><\/span><\/pre>\n\n\n<p><strong>handler.py:<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-4\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">from selenium import webdriver\nfrom selenium.webdriver.chrome.options import Options\nfrom selenium.webdriver.common.by import By\n\ndef handler(event, context):\n    options = Options()\n    options.add_argument(<span class=\"hljs-string\">\"--headless=new\"<\/span>)   <span class=\"hljs-comment\"># Updated: Chrome 112+ syntax<\/span>\n    options.add_argument(<span class=\"hljs-string\">\"--no-sandbox\"<\/span>)\n    options.add_argument(<span class=\"hljs-string\">\"--disable-dev-shm-usage\"<\/span>)\n    options.add_argument(<span class=\"hljs-string\">\"--single-process\"<\/span>)\n    options.add_argument(<span class=\"hljs-string\">\"--disable-gpu\"<\/span>)\n    options.add_argument(<span class=\"hljs-string\">\"--window-size=1920,1080\"<\/span>)\n\n    driver = webdriver.Chrome(options=options)\n\n    <span class=\"hljs-keyword\">try<\/span>:\n        driver.get(event.get(<span class=\"hljs-string\">\"url\"<\/span>, <span class=\"hljs-string\">\"https:\/\/example.com\"<\/span>))\n        title = driver.title\n        header = driver.find_element(By.CSS_SELECTOR, <span class=\"hljs-string\">'h1'<\/span>).text\n        <span class=\"hljs-keyword\">return<\/span> {<span class=\"hljs-string\">\"title\"<\/span>: title, <span class=\"hljs-string\">\"header\"<\/span>: header}\n    <span class=\"hljs-keyword\">finally<\/span>:\n        driver.quit()  <span class=\"hljs-comment\"># Always close the browser<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-4\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">Principais mudan\u00e7as em rela\u00e7\u00e3o ao original:<\/h3>\n\n\n\n<p>A flag <code><strong>--headless<\/strong><\/code> foi depreciada no Chrome 112. A nova flag \u00e9 <code><strong>--headless=new<\/strong><\/code>. Usar a flag antiga gera avisos de compatibilidade e pode causar problemas de renderiza\u00e7\u00e3o em vers\u00f5es modernas do Chrome. Em 2026, use sempre <strong><code>--headless=new<\/code> <\/strong>no seu setup.<\/p>\n\n\n\n<p>O bloco <code><strong>try\/finally<\/strong><\/code> em volta do <code><strong>driver.quit()<\/strong><\/code> tamb\u00e9m importa: sem ele, um erro no scraping deixa um processo do Chrome rodando dentro do container do Lambda, consumindo mem\u00f3ria at\u00e9 o fim da invoca\u00e7\u00e3o.<\/p>\n\n\n\n<p><strong>serverless.yml (Serverless Framework v4):<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-5\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">service: selenium-scraper\n\nprovider:\n  name: aws\n  region: us-east<span class=\"hljs-number\">-1<\/span>\n  timeout: <span class=\"hljs-number\">900<\/span>  <span class=\"hljs-comment\"># 15 minutes maximum<\/span>\n\nfunctions:\n  scraper:\n    image:\n      uri: &lt;your-ecr-image-uri&gt;\n    memorySize: <span class=\"hljs-number\">2048<\/span>  <span class=\"hljs-comment\"># Chrome needs memory, 2GB is a safe starting point<\/span>\n    events:\n      - schedule: rate(<span class=\"hljs-number\">6<\/span> hours)<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-5\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Build, push e deploy:<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-6\" data-shcb-language-name=\"HTML, XML\" data-shcb-language-slug=\"xml\"><span><code class=\"hljs language-xml\"># Build and push the container to ECR\naws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">account-id<\/span>&gt;<\/span>.dkr.ecr.us-east-1.amazonaws.com\ndocker build -t selenium-scraper .\ndocker tag selenium-scraper:latest <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">account-id<\/span>&gt;<\/span>.dkr.ecr.us-east-1.amazonaws.com\/selenium-scraper:latest\ndocker push <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">account-id<\/span>&gt;<\/span>.dkr.ecr.us-east-1.amazonaws.com\/selenium-scraper:latest\n\n# Deploy with Serverless Framework\nserverless deploy<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-6\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">HTML, XML<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">xml<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Observa\u00e7\u00e3o:<\/strong> fun\u00e7\u00f5es Lambda baseadas em container n\u00e3o podem ser gerenciadas pela interface do dashboard do Serverless, mas os logs ficam acess\u00edveis pelo AWS CloudWatch. Monitore l\u00e1 o tempo de execu\u00e7\u00e3o e o uso de mem\u00f3ria do seu scraper: o Chrome consome muita mem\u00f3ria, e o Lambda encerra fun\u00e7\u00f5es que passam do teto configurado.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Quando usar Playwright no lugar<\/h3>\n\n\n\n<p>Se voc\u00ea est\u00e1 come\u00e7ando um projeto de scraping novo em 2026, o Playwright \u00e9 o padr\u00e3o mais forte. As vantagens pr\u00e1ticas s\u00e3o reais:<\/p>\n\n\n\n<p><strong>A espera autom\u00e1tica elimina uma classe inteira de bugs.<\/strong> A causa mais comum de scraper inst\u00e1vel \u00e9 timing: um elemento ainda n\u00e3o est\u00e1 vis\u00edvel, uma requisi\u00e7\u00e3o de rede n\u00e3o terminou, uma anima\u00e7\u00e3o em JavaScript ainda est\u00e1 rodando. O Selenium exige chamadas expl\u00edcitas de <code>WebDriverWait<\/code> em cada intera\u00e7\u00e3o. Se voc\u00ea esquecer uma, o scraper quebra de forma intermitente e o debug vira um sofrimento. O Playwright espera automaticamente at\u00e9 que o elemento esteja pronto para receber a intera\u00e7\u00e3o. S\u00f3 isso j\u00e1 remove a maioria das falhas ligadas a timing.<\/p>\n\n\n\n<p><strong>O suporte a async permite um throughput muito maior.<\/strong> A API ass\u00edncrona do Playwright torna o scraping concorrente direto. Coletar 20 URLs em paralelo s\u00e3o poucas linhas com <code>asyncio<\/code>. No Selenium, a mesma tarefa exige gerenciar threads e \u00e9 bem mais dif\u00edcil de depurar.<\/p>\n\n\n\n<p><strong>A configura\u00e7\u00e3o \u00e9 mais simples.<\/strong> Sem casar vers\u00e3o de ChromeDriver. Sem matriz de compatibilidade entre vers\u00f5es do Chrome e do ChromeDriver. O <code>playwright install<\/code> baixa os bin\u00e1rios corretos do navegador automaticamente e mant\u00e9m tudo sincronizado.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Playwright no Lambda:<\/h3>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-7\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\"><span class=\"hljs-keyword\">import<\/span> asyncio\n<span class=\"hljs-keyword\">from<\/span> playwright.async_api <span class=\"hljs-keyword\">import<\/span> async_playwright\n\n<span class=\"hljs-keyword\">async<\/span> def scrape(url: str) -&gt; dict:\n    <span class=\"hljs-keyword\">async<\/span> <span class=\"hljs-keyword\">with<\/span> async_playwright() <span class=\"hljs-keyword\">as<\/span> p:\n        browser = <span class=\"hljs-keyword\">await<\/span> p.chromium.launch(\n            headless=True,\n            args=&#91;<span class=\"hljs-string\">\"--no-sandbox\"<\/span>, <span class=\"hljs-string\">\"--disable-dev-shm-usage\"<\/span>]\n        )\n        page = <span class=\"hljs-keyword\">await<\/span> browser.new_page()\n\n        <span class=\"hljs-attr\">try<\/span>:\n            <span class=\"hljs-keyword\">await<\/span> page.goto(url, wait_until=<span class=\"hljs-string\">\"networkidle\"<\/span>)\n            title = <span class=\"hljs-keyword\">await<\/span> page.title()\n            header = <span class=\"hljs-keyword\">await<\/span> page.locator(<span class=\"hljs-string\">\"h1\"<\/span>).text_content()\n            <span class=\"hljs-keyword\">return<\/span> {<span class=\"hljs-string\">\"title\"<\/span>: title, <span class=\"hljs-string\">\"header\"<\/span>: header}\n        <span class=\"hljs-attr\">finally<\/span>:\n            <span class=\"hljs-keyword\">await<\/span> browser.close()\n\ndef handler(event, context):\n    url = event.get(<span class=\"hljs-string\">\"url\"<\/span>, <span class=\"hljs-string\">\"https:\/\/example.com\"<\/span>)\n    <span class=\"hljs-keyword\">return<\/span> asyncio.run(scrape(url))<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-7\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>A estrutura do Dockerfile para Playwright no Lambda \u00e9 parecida: fun\u00e7\u00e3o em container com o Playwright e os bin\u00e1rios de navegador instalados. O comando <code>playwright install chromium<\/code> durante o build da imagem cuida de todo o gerenciamento de bin\u00e1rios.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Quando Selenium ainda faz sentido<\/h3>\n\n\n\n<p>Reescrever c\u00f3digo Selenium que funciona n\u00e3o se justifica com \u201co Playwright \u00e9 mais novo\u201d. Se voc\u00ea tem uma base de c\u00f3digo Selenium que roda de forma confi\u00e1vel, o custo da migra\u00e7\u00e3o \u00e9 real e o ganho \u00e9 marginal para scrapers est\u00e1veis.<\/p>\n\n\n\n<p>Fique com o Selenium quando a base de c\u00f3digo j\u00e1 est\u00e1 estabelecida e funcionando, quando seu time investiu em Selenium Grid para execu\u00e7\u00e3o distribu\u00edda, ou quando voc\u00ea precisa de Java, C# ou Ruby. Nessas linguagens, os bindings do Playwright s\u00e3o menos maduros que em Python e JavaScript.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Detec\u00e7\u00e3o de bots<\/h3>\n\n\n\n<p>Tanto Selenium quanto Playwright s\u00e3o detect\u00e1veis por sistemas anti-bot modernos. Cloudflare, Akamai, PerimeterX e DataDome fazem fingerprint de ferramentas de automa\u00e7\u00e3o de navegador em n\u00edvel de rede e de navegador: inspecionam headers, propriedades de JavaScript, padr\u00f5es de movimento do mouse e assinaturas de timing.<\/p>\n\n\n\n<p>O Selenium puro \u00e9 especialmente f\u00e1cil de detectar porque define <code>navigator.webdriver = true<\/code> por padr\u00e3o. Wrappers de stealth como o <code>undetected-chromedriver<\/code> corrigem isso, mas os sistemas de detec\u00e7\u00e3o evoluem e esses patches exigem manuten\u00e7\u00e3o.<\/p>\n\n\n\n<p>Para alvos com<strong> prote\u00e7\u00e3o anti-bot s\u00e9ria<\/strong>, infraestrutura de navegador gerenciada \u00e9 a escolha mais pragm\u00e1tica. Browserless, Scrapfly e Apify cuidam de pools de navegadores, rota\u00e7\u00e3o de proxy e antidetec\u00e7\u00e3o no n\u00edvel de infraestrutura. O custo por requisi\u00e7\u00e3o \u00e9 maior do que rodar o seu pr\u00f3prio Lambda, mas a confiabilidade \u00e9 bem melhor e o esfor\u00e7o de manuten\u00e7\u00e3o \u00e9 bem menor.<\/p>\n\n\n\n<p>Para alvos sem prote\u00e7\u00e3o, como ferramentas internas e dados de acesso p\u00fablico sem detec\u00e7\u00e3o de bot, um setup simples de Selenium ou Playwright no Lambda funciona de forma confi\u00e1vel.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p><strong>Leia tamb\u00e9m: <\/strong><a href=\"https:\/\/cheesecakelabs.com\/blog\/harness-engineering-quando-esta-realmente-pronto\/\" id=\"13828\" target=\"_blank\" rel=\"noreferrer noopener\">Harness engineering: por que \u201cpronto\u201d n\u00e3o \u00e9 o agent dizer que terminou<\/a><\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"Conclusions\"><strong>Para fechar<\/strong><\/h2>\n\n\n\n<p>Este \u00e9 um exemplo bem espec\u00edfico de uso do AWS Lambda com Selenium, mas espero que ele mostre o potencial dessas tecnologias. No lugar de um web scraper, d\u00e1 para criar fun\u00e7\u00f5es que rodam testes end-to-end e mandam uma mensagem no Slack em caso de falha. Ou uma API que calcula a dist\u00e2ncia entre duas strings e devolve o resultado na resposta HTTP. O limite \u00e9 a sua imagina\u00e7\u00e3o.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"http:\/\/cheesecakelabs.com\/services\" target=\"_blank\" rel=\" noreferrer noopener\"><img decoding=\"async\" width=\"1200\" height=\"409\" src=\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/06\/legacy-app-ckl-1200x409.jpg\" alt=\"legacy-app-ckl | | Cheesecake Labs\" class=\"wp-image-13491\" srcset=\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/06\/legacy-app-ckl-1200x409.jpg 1200w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/06\/legacy-app-ckl-600x205.jpg 600w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/06\/legacy-app-ckl-768x262.jpg 768w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/06\/legacy-app-ckl-1536x524.jpg 1536w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/06\/legacy-app-ckl-760x259.jpg 760w, https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/06\/legacy-app-ckl.jpg 1920w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/a><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Web scraping pode economizar v\u00e1rias horas em compara\u00e7\u00e3o com a coleta manual de dados em sites, e o AWS Lambda \u00e9 uma boa forma de configurar scripts para rodar sob demanda. Neste artigo, voc\u00ea vai ver o que precisa saber sobre web scraping e AWS Lambda, com o que cada um tem de bom e [&hellip;]<\/p>\n","protected":false},"author":92,"featured_media":11064,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_yoast_wpseo_title":"Como usar Selenium para web scraping no AWS Lambda","_yoast_wpseo_metadesc":"Como usar Selenium para web scraping no AWS Lambda: quando o scraping se justifica, t\u00e9cnicas essenciais e por que rodar tudo sob demanda.","_yoast_wpseo_meta-robots-noindex":"","_yoast_wpseo_canonical":"","ai_summary":"Web scraping deve ser tratado como \u00faltimo recurso: se existe API, use a API; o scraping se justifica quando n\u00e3o h\u00e1 API, os dados s\u00e3o p\u00fablicos e a estrutura do site \u00e9 est\u00e1vel. Para conte\u00fado est\u00e1tico, requests\/urllib com BeautifulSoup basta; p\u00e1ginas renderizadas por JavaScript exigem automa\u00e7\u00e3o de navegador com Selenium ou Playwright.\nEntre as op\u00e7\u00f5es da AWS (EC2, Fargate e Lambda), o Lambda \u00e9 o servi\u00e7o serverless mais barato, com limita\u00e7\u00f5es de timeout de 15 minutos e pacote de deploy de 250 MB (at\u00e9 10 GB com containers); como o bin\u00e1rio do Chrome tem cerca de 500 MB, \u00e9 necess\u00e1rio usar a abordagem de container.\nO projeto de exemplo usa Serverless Framework, um Dockerfile que instala Chrome e ChromeDriver, e um handler em Python configurado para rodar headless e em processo \u00fanico, disparado por cronjob a cada 6 horas.\nO setup atualizado para 2026 usa Python 3.12, Serverless Framework v4, mem\u00f3ria de 2048 MB, a flag --headless=new (a antiga --headless foi depreciada no Chrome 112) e um bloco try\/finally em volta do driver.quit() para evitar processos do Chrome \u00f3rf\u00e3os consumindo mem\u00f3ria.","ai_summary_en":"","ai_summary_pt-br":"Web scraping deve ser tratado como \u00faltimo recurso: se existe API, use a API; o scraping se justifica quando n\u00e3o h\u00e1 API, os dados s\u00e3o p\u00fablicos e a estrutura do site \u00e9 est\u00e1vel. Para conte\u00fado est\u00e1tico, requests\/urllib com BeautifulSoup basta; p\u00e1ginas renderizadas por JavaScript exigem automa\u00e7\u00e3o de navegador com Selenium ou Playwright.\nEntre as op\u00e7\u00f5es da AWS (EC2, Fargate e Lambda), o Lambda \u00e9 o servi\u00e7o serverless mais barato, com limita\u00e7\u00f5es de timeout de 15 minutos e pacote de deploy de 250 MB (at\u00e9 10 GB com containers); como o bin\u00e1rio do Chrome tem cerca de 500 MB, \u00e9 necess\u00e1rio usar a abordagem de container.\nO projeto de exemplo usa Serverless Framework, um Dockerfile que instala Chrome e ChromeDriver, e um handler em Python configurado para rodar headless e em processo \u00fanico, disparado por cronjob a cada 6 horas.\nO setup atualizado para 2026 usa Python 3.12, Serverless Framework v4, mem\u00f3ria de 2048 MB, a flag --headless=new (a antiga --headless foi depreciada no Chrome 112) e um bloco try\/finally em volta do driver.quit() para evitar processos do Chrome \u00f3rf\u00e3os consumindo mem\u00f3ria.","footnotes":"","ckl_wpml_lang":"br","ckl_wpml_source_id":11057,"ckl_wpml_status":"ok: lang=br trid=53893 source=11057 at 2026-08-14 18:03:46"},"categories":[1423,432,471],"tags":[305,1148],"class_list":["post-14618","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-application-modernization","category-engineering","category-engenharia","tag-tag-development","tag-tag-development-br"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.1.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Como usar Selenium para web scraping no AWS Lambda<\/title>\n<meta name=\"description\" content=\"Como usar Selenium para web scraping no AWS Lambda: quando o scraping se justifica, t\u00e9cnicas essenciais e por que rodar tudo sob demanda.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/\" \/>\n<meta property=\"og:locale\" content=\"pt_BR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Como usar Selenium para web scraping no AWS Lambda\" \/>\n<meta property=\"og:description\" content=\"Como usar Selenium para web scraping no AWS Lambda: quando o scraping se justifica, t\u00e9cnicas essenciais e por que rodar tudo sob demanda.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/\" \/>\n<meta property=\"og:site_name\" content=\"Cheesecake Labs\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cheesecakelabs\" \/>\n<meta property=\"article:published_time\" content=\"2023-01-12T20:42:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-08-15T07:05:52+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"860\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Cheesecake Labs\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@cheesecakelabs\" \/>\n<meta name=\"twitter:site\" content=\"@cheesecakelabs\" \/>\n<meta name=\"twitter:label1\" content=\"Escrito por\" \/>\n\t<meta name=\"twitter:data1\" content=\"\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. tempo de leitura\" \/>\n\t<meta name=\"twitter:data2\" content=\"14 minutos\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/\"},\"author\":{\"name\":\"Diana Martins\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#\/schema\/person\/37fa2f0fa5f8d3cbaeb21b973fa5c4b9\"},\"headline\":\"Como usar Selenium para web scraping no AWS Lambda\",\"datePublished\":\"2023-01-12T20:42:29+00:00\",\"dateModified\":\"2026-08-15T07:05:52+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/\"},\"wordCount\":2453,\"publisher\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#organization\"},\"image\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg\",\"keywords\":[\"development\",\"development\"],\"articleSection\":[\"Application Modernization\",\"Engineering\",\"Engenharia\"],\"inLanguage\":\"pt-BR\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/\",\"url\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/\",\"name\":\"Como usar Selenium para web scraping no AWS Lambda\",\"isPartOf\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg\",\"datePublished\":\"2023-01-12T20:42:29+00:00\",\"dateModified\":\"2026-08-15T07:05:52+00:00\",\"description\":\"Como usar Selenium para web scraping no AWS Lambda: quando o scraping se justifica, t\u00e9cnicas essenciais e por que rodar tudo sob demanda.\",\"breadcrumb\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#breadcrumb\"},\"inLanguage\":\"pt-BR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"pt-BR\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage\",\"url\":\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg\",\"contentUrl\":\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg\",\"width\":1920,\"height\":860,\"caption\":\"woman programming in front of a pc\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/cheesecakelabs.com\/blog\/br\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Como usar Selenium para web scraping no AWS Lambda\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#website\",\"url\":\"https:\/\/cheesecakelabs.com\/blog\/br\/\",\"name\":\"Cheesecake Labs\",\"description\":\"Empresa de desenvolvimento e design de aplicativos mobile &amp; web que est\u00e1 reinventando o desenvolvimento de produtos com times remotos. N\u00f3s desenvolvemos aplicativos iOS, Android e aplica\u00e7\u00f5es Web com as melhores empresas dos EUA, do Brasil e do mundo.\",\"publisher\":{\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/cheesecakelabs.com\/blog\/br\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"pt-BR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#organization\",\"name\":\"Cheesecake Labs\",\"alternateName\":\"Cheesecake Labs Inc\",\"url\":\"https:\/\/cheesecakelabs.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cheesecakelabs.com\/#logo\",\"url\":\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2022\/06\/cheesecake-labs-1.png\",\"contentUrl\":\"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2022\/06\/cheesecake-labs-1.png\",\"caption\":\"Cheesecake Labs\",\"inLanguage\":\"br\"},\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/cheesecakelabs.com\/#primary-image\",\"url\":\"https:\/\/ckl-website-v4-strapi-prod.s3.us-east-2.amazonaws.com\/ai_software_development_company_83fb512983.webp\",\"contentUrl\":\"https:\/\/ckl-website-v4-strapi-prod.s3.us-east-2.amazonaws.com\/ai_software_development_company_83fb512983.webp\",\"width\":1920,\"height\":1080,\"caption\":\"Cheesecake Labs \u2014 AI, Data & Blockchain software development services\",\"inLanguage\":\"br\"},\"sameAs\":[\"https:\/\/www.facebook.com\/cheesecakelabs\",\"https:\/\/x.com\/cheesecakelabs\",\"https:\/\/www.instagram.com\/cheesecakelabs\/\",\"https:\/\/www.linkedin.com\/company\/cheesecake-labs\/\",\"https:\/\/www.youtube.com\/channel\/UCdGEQ5AHJcmIlaOaI5fGGVA\",\"https:\/\/clutch.co\/profile\/cheesecake-labs\",\"https:\/\/www.behance.net\/cheesecakelabs\",\"https:\/\/dribbble.com\/cheesecakelabs\",\"https:\/\/www.designrush.com\/agency\/profile\/cheesecake-labs\",\"https:\/\/www.g2.com\/products\/cheesecake-labs\/reviews\"],\"description\":\"Cheesecake Labs is a software development studio that designs and builds custom digital products \u2014 web, mobile, and platforms \u2014 combining product design and high-performance engineering.\",\"foundingDate\":\"2013\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#\/schema\/person\/37fa2f0fa5f8d3cbaeb21b973fa5c4b9\",\"name\":\"Diana Martins\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"pt-BR\",\"@id\":\"https:\/\/cheesecakelabs.com\/blog\/br\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d074fb633a22723051c9d98ffc13065a28f0b52bd30a4c0a832ee19820caa4ba?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d074fb633a22723051c9d98ffc13065a28f0b52bd30a4c0a832ee19820caa4ba?s=96&d=mm&r=g\",\"caption\":\"Diana Martins\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Como usar Selenium para web scraping no AWS Lambda","description":"Como usar Selenium para web scraping no AWS Lambda: quando o scraping se justifica, t\u00e9cnicas essenciais e por que rodar tudo sob demanda.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/","og_locale":"pt_BR","og_type":"article","og_title":"Como usar Selenium para web scraping no AWS Lambda","og_description":"Como usar Selenium para web scraping no AWS Lambda: quando o scraping se justifica, t\u00e9cnicas essenciais e por que rodar tudo sob demanda.","og_url":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/","og_site_name":"Cheesecake Labs","article_publisher":"https:\/\/www.facebook.com\/cheesecakelabs","article_published_time":"2023-01-12T20:42:29+00:00","article_modified_time":"2026-08-15T07:05:52+00:00","og_image":[{"width":1920,"height":860,"url":"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg","type":"image\/jpeg"}],"author":"Cheesecake Labs","twitter_card":"summary_large_image","twitter_creator":"@cheesecakelabs","twitter_site":"@cheesecakelabs","twitter_misc":{"Escrito por":null,"Est. tempo de leitura":"14 minutos"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#article","isPartOf":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/"},"author":{"name":"Diana Martins","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#\/schema\/person\/37fa2f0fa5f8d3cbaeb21b973fa5c4b9"},"headline":"Como usar Selenium para web scraping no AWS Lambda","datePublished":"2023-01-12T20:42:29+00:00","dateModified":"2026-08-15T07:05:52+00:00","mainEntityOfPage":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/"},"wordCount":2453,"publisher":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#organization"},"image":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage"},"thumbnailUrl":"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg","keywords":["development","development"],"articleSection":["Application Modernization","Engineering","Engenharia"],"inLanguage":"pt-BR"},{"@type":"WebPage","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/","url":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/","name":"Como usar Selenium para web scraping no AWS Lambda","isPartOf":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#website"},"primaryImageOfPage":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage"},"image":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage"},"thumbnailUrl":"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg","datePublished":"2023-01-12T20:42:29+00:00","dateModified":"2026-08-15T07:05:52+00:00","description":"Como usar Selenium para web scraping no AWS Lambda: quando o scraping se justifica, t\u00e9cnicas essenciais e por que rodar tudo sob demanda.","breadcrumb":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#breadcrumb"},"inLanguage":"pt-BR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/"]}]},{"@type":"ImageObject","inLanguage":"pt-BR","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#primaryimage","url":"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg","contentUrl":"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2023\/01\/selenium-web-scraping-aws-lambda.jpg","width":1920,"height":860,"caption":"woman programming in front of a pc"},{"@type":"BreadcrumbList","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/selenium-web-scraping-aws-lambda\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/cheesecakelabs.com\/blog\/br\/blog\/"},{"@type":"ListItem","position":2,"name":"Como usar Selenium para web scraping no AWS Lambda"}]},{"@type":"WebSite","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#website","url":"https:\/\/cheesecakelabs.com\/blog\/br\/","name":"Cheesecake Labs","description":"Empresa de desenvolvimento e design de aplicativos mobile &amp; web que est\u00e1 reinventando o desenvolvimento de produtos com times remotos. N\u00f3s desenvolvemos aplicativos iOS, Android e aplica\u00e7\u00f5es Web com as melhores empresas dos EUA, do Brasil e do mundo.","publisher":{"@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cheesecakelabs.com\/blog\/br\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"pt-BR"},{"@type":"Organization","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#organization","name":"Cheesecake Labs","alternateName":"Cheesecake Labs Inc","url":"https:\/\/cheesecakelabs.com\/","logo":{"@type":"ImageObject","@id":"https:\/\/cheesecakelabs.com\/#logo","url":"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2022\/06\/cheesecake-labs-1.png","contentUrl":"https:\/\/ckl-website-static.s3.amazonaws.com\/wp-content\/uploads\/2022\/06\/cheesecake-labs-1.png","caption":"Cheesecake Labs","inLanguage":"br"},"image":{"@type":"ImageObject","@id":"https:\/\/cheesecakelabs.com\/#primary-image","url":"https:\/\/ckl-website-v4-strapi-prod.s3.us-east-2.amazonaws.com\/ai_software_development_company_83fb512983.webp","contentUrl":"https:\/\/ckl-website-v4-strapi-prod.s3.us-east-2.amazonaws.com\/ai_software_development_company_83fb512983.webp","width":1920,"height":1080,"caption":"Cheesecake Labs \u2014 AI, Data & Blockchain software development services","inLanguage":"br"},"sameAs":["https:\/\/www.facebook.com\/cheesecakelabs","https:\/\/x.com\/cheesecakelabs","https:\/\/www.instagram.com\/cheesecakelabs\/","https:\/\/www.linkedin.com\/company\/cheesecake-labs\/","https:\/\/www.youtube.com\/channel\/UCdGEQ5AHJcmIlaOaI5fGGVA","https:\/\/clutch.co\/profile\/cheesecake-labs","https:\/\/www.behance.net\/cheesecakelabs","https:\/\/dribbble.com\/cheesecakelabs","https:\/\/www.designrush.com\/agency\/profile\/cheesecake-labs","https:\/\/www.g2.com\/products\/cheesecake-labs\/reviews"],"description":"Cheesecake Labs is a software development studio that designs and builds custom digital products \u2014 web, mobile, and platforms \u2014 combining product design and high-performance engineering.","foundingDate":"2013"},{"@type":"Person","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#\/schema\/person\/37fa2f0fa5f8d3cbaeb21b973fa5c4b9","name":"Diana Martins","image":{"@type":"ImageObject","inLanguage":"pt-BR","@id":"https:\/\/cheesecakelabs.com\/blog\/br\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d074fb633a22723051c9d98ffc13065a28f0b52bd30a4c0a832ee19820caa4ba?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d074fb633a22723051c9d98ffc13065a28f0b52bd30a4c0a832ee19820caa4ba?s=96&d=mm&r=g","caption":"Diana Martins"}}]}},"_links":{"self":[{"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/posts\/14618","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/users\/92"}],"replies":[{"embeddable":true,"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/comments?post=14618"}],"version-history":[{"count":3,"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/posts\/14618\/revisions"}],"predecessor-version":[{"id":15049,"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/posts\/14618\/revisions\/15049"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/media\/11064"}],"wp:attachment":[{"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/media?parent=14618"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/categories?post=14618"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cheesecakelabs.com\/blog\/br\/wp-json\/wp\/v2\/tags?post=14618"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}