I am trying to fetch and parse the JSON returned by the below, in Nodejs. It seems the problem is that the node code is trying to parse the JSON string which contains newline characters, but I am not sure how to avoid this, and why fetch works in chrome devtools. I'm guessing the problem is my simplistic knowledge of https requests, so I would really appreciate it if somebody can explain what is going wrong.
const url = "https://beta.charitycommission.gov.uk/umbraco/api/charityApi/getSearchResults?searchText=&pageNumber=1&contextId=1126&onlyShow=&&&&&&&"
I can successfully run this in the chrome devtools open on the domain. Also, the JSON appears to automatically get parsed to an object, which I didn't expect.
fetch(url).then(res => res.json()).then(json => console.log(json.pageItems))
I have tried the following in node, but none work.
const fetch = require("node-fetch");
await fetch(url).then(res => res.json()); // Unexpected token in JSON at position 0
const rp = require('request-promise-native');
const json = await rp({uri: url}) // returns JSON string with newline characters
JSON.parse(json) // Unexpected token in JSON at position 0
The problem is that your JSON file is saved in UTF-8 BOM format.
What does it mean?
Your file begins with a so-called Byte Order Mark character, U+FEFF (Zero Width No-break Space).
That's invisible (has zero width), but still there, and the JSON interpreter fails to parse it.
How to solve?
Use res.text() and then JSON.parse(text.slice(1)) to remove the leading character:
fetch(url)
.then(res => res.text())
.then(text => JSON.parse(text.slice(1)))
.then(json => console.log(json.pageItems))
Why does it work in Chrome?
Chrome automatically removes the BOM character from any fetched/downloaded file, to avoid similar issues.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments