Btukfyl

I scrape a jobs portal using this code:

const puppeteer = require('puppeteer')



export default function scrape() {

  ;(async () => {

    const browser = await puppeteer.launch()

    const page = await browser.newPage()

    await page.goto('https://www.example.de/jobs/javascript')



    const position = await page.evaluate(() =>

      Array.from(document.querySelectorAll('h2')).map(

        position => position.innerText

      )

    )



    // const logo = await page.evaluate(() =>

    //   Array.from(document.querySelectorAll('div.job-element__logo img')).map(

    //     logo => logo.src

    //   )

    // )



    console.log(JSON.stringify(position))



    await browser.close()

  })()

}

I pasted a example URL here, of course I use a real one in my example. Otherwise the code should work I thought.

My question:

I can console log my extracted data under the const position. That works! But I would like to reuse it now in another react component. Which does not work...

I tried to export it as a function and import it in my component called JobCard.js. But it prints a warning saying:

Can't resolve './scraper' in .../components'.

Here is a screenshot of my data tree: My data structure

If I put the scraper.js into components it says:

./node_modules/puppeteer/lib/WebSocketTransport.js
Module not found: Can't resolve 'ws' in '/remote-jobs-app/node_modules/puppeteer/lib'

I just want to reuse the const position in another component. But I cannot get it done. Any ideas maybe?

you can't import your scrapper component in react.js component
when you scrapping you are running node.js. that is not reactjs
it's a server side thing .

When you running your react.js it's a browser based thing.
i suggest you that, you scrape data and save it a file like jobdata.json and then import this in your component
then used that data

If you want dynamic result, you need a server side code. So overall design is like
you scrape the data, save it in database. make a api that and use react to show that