温馨提示:本文翻译自stackoverflow.com,查看原文请点击:其他 - Puppeteer with Crawlera Proxy
puppeteer

其他 - 具有Crawlera代理的木偶

发布于 2020-04-17 14:23:23

我无法通过具有身份验证的代理发出p请求。

尝试了两种代理URL身份验证: --proxy-server=u:p@proxy.crawlera.com:8010

还有木偶戏 page.authenticate(u,p)

仍在 ERR_NO_SUPPORTED_PROXIES

我的代码:

require('dotenv').config();
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    ignoreHTTPSErrors: true,
    args: ['--proxy-server=proxy.crawlera.com:8010']
  });

  const page = await browser.newPage();
  await page.setExtraHTTPHeaders({
    'Proxy-Authorization':
      'Basic ' +
      Buffer.from(`${process.env.CRAWLERA_APIKEY}:`).toString('base64')
  });

  page.on('console', (...args) => console.log('PAGE LOG:', ...args));
  const path = `https://www.andersonassociates.net/`;
  await page.setViewport({ width: 1680, height: 895 });

  try {
    console.log('before-goto', path);
    var start = +new Date();
    var resp = await page.goto(path, {
      timeout: 0,
      waitUntil: 'domcontentloaded'
    });

    console.log('after-goto', path);
    var end = +new Date();
    console.log('start-end-diff', (end - start) / 1000);

    if (!resp.ok) {
      browser.close();
      return { status: resp.status, error: `ASIN NOT OK. ${resp.status}` };
    }
    console.log('goto', path);
  } catch (error) {
    console.log('page.goto ERROR', error.stack.split('\n'));
    browser.close();
    return { error: error.toString(), stack: error.stack.split('\n') };
  }

  try {
    await page.screenshot({ path: `tmp/anderson.png`, fullPage: true });
    console.log('screenshot');
    browser.close();
  } catch (e) {
    browser.close();
    console.log('screenshot error', e.stack.split('\n'));
  }
})();

更新!

查看更多

提问者
Blair Anderson
被浏览
24
Agung Darmanto 2018-02-07 12:53

我也使用了Crawlera作为代理服务,已经使用了代理链和page.authenticate方法,但是没有运气,我认为这是因为Crawlera提供了一个空密码,我通过使用解决了page.setExtraHTTPHeaders

const browser = await puppeteer.launch({
    ignoreHTTPSErrors: true, // To allow https url
    args: ['--proxy-server=proxy.crawlera.com:8010']
});
const page = await browser.newPage();
await page.setExtraHTTPHeaders({
    'Proxy-Authorization': 'Basic ' + Buffer.from('<APIKEY>:').toString('base64'),
});

希望这可以帮助。