Help me with my Web-storm code for web scraper


Joined
Jan 31, 2022
Messages
1
Reaction score
0
JSON:
{
  "name": "web-scraper",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "nodemon index.js",
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "axios": "^0.21.4",
    "cheerio": "^1.0.0-rc.10",
    "cors": "^2.8.5",
    "express": "^4.17.1"
  }
[QUOTE]
}
Code:
const PORT = 8000
const axios = require('axios')
const express = require('express')
const app = express()
const cors = require('cors')
app.use(cors())

const url = 'https://www.theguardian.com/uk'

app.get('/', function (req, res) {
    res.json('This is my webscraper')
})

app.get('/results', (req, res) => {
    axios(url)
        .then(response => {
            const html = response.data
            const $ = cheerio.load(html)
            const articles = []

            $('.fc-item__title', html).each(function () { //<-- cannot be a function expression
                const title = $(this).text()
                const url = $(this).find('a').attr('href')
                articles.push({
                    title,
                    url
                })
            })
            res.json(articles)
        }).catch(err => console.log(err))

})


app.listen(PORT, () => console.log(`server running on PORT ${PORT}`))
[/QUOTE]
 
Ad

Advertisements

Joined
Mar 3, 2021
Messages
243
Reaction score
29
Hello! First experience with node, that was pretty neat. The only problem I had running this server was that cheerio was undefined, so I had to add const cheerio = require('cheerio') to the top. I tested with Node v14.18.1 and npm installed all of the packages.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Top