I have to write a Jupyter Notebook to perform an analysis of the Q&A threads from the category Profile. Data to analyze are contained in a JSON file. This file is vey big and it contains more than one discussion. When I import the file and i try to visualize it in the terminal, i visualize this error:
JSONDecodeError: Extra data: line 855 column 1 (char 26418)
Analyzing the imported file I noticed that line 855 corresponds with the end of a discussion and the beginning of the next one.
Following a part of the json file I'm talking about.
{
"Title": "How to get a new badge?",
"Number": "18294",
"Category": "Profile",
"Author": "deeperwhales",
"Date": "2022-06-10T18:42:30Z",
"State": "Answered",
"Answered_by": "wavescats",
"Body": "How to get a new badge?",
"Upvotes": 140,
"Labels": [
"Profile"
],
"Participants": 31,
"Answer_count": 80,
"Reply_count": 502,
"Answers": [
{
"IsOffTopic": false,
"Author": "wavescats",
"Date": "2022-06-10T18:43:14Z",
"Body": "After answering two discussions, You will get Galaxy Brain badge More details here: https://github.com/Schweinepriester/github-profile-achievements",
"Upvotes": 59,
"Accepted": true,
"Reply_count": 239,
"Replies": [
{
"Author": "pajeeh",
"Body": "Use the most number of languages.",
"Date": "2022-10-08T09:06:56Z",
"IsAuthor": false,
"Sentiment": "neutral"
},
{
"Author": "Khairul989",
"Body": "thanks",
"Date": "2022-10-09T16:05:54Z",
"IsAuthor": false,
"Sentiment": "positive"
},
{
"Author": "ibrahimmemonn",
"Body": "Thanks",
"Date": "2022-10-18T10:10:48Z",
"IsAuthor": false,
"Sentiment": "positive"
}
],
"Sentiment": "positive"
},
{
"IsOffTopic": false,
"Author": "akbar-ardiansyah",
"Date": "2022-06-10T19:44:45Z",
"Body": "pull shrark was opened when you opened pull requests that have been merged.",
"Upvotes": 6,
"Accepted": false,
"Reply_count": 16,
"Replies": [
{
"Author": "deividepaulino1",
"Body": "thanks",
"Date": "2022-07-08T19:33:35Z",
"IsAuthor": false,
"Sentiment": "positive"
},
{
"Author": "darkhorse-coder",
"Body": "Exactly, if you approach 100 pr merged, you will get Silver Pull Shark. ;)",
"Date": "2022-07-20T16:55:13Z",
"IsAuthor": false,
"Sentiment": "neutral"
},
{
"Author": "Splayfery",
"Body": "How can I get different levels of this achievment?",
"Date": "2022-07-20T18:24:17Z",
"IsAuthor": false,
"Sentiment": "neutral"
},
{
"Author": "wizardigor",
"Body": "Quais outros emblemas est\u00e3o disponiveis?",
"Date": "2022-08-25T13:08:52Z",
"IsAuthor": false,
"Sentiment": "neutral"
},
{
"Author": "burhancan-stack",
"Body": "thanks.",
"Date": "2022-09-20T12:09:56Z",
"IsAuthor": false,
"Sentiment": "positive"
}
],
"Sentiment": "neutral"
}
],
"Sentiment": "neutral"
} ********************************Line 855*********************************
{
"Title": "feed back on achievement badges",
"Number": "21073",
"Category": "Profile",
"Author": "SteveALee",
"Date": "2022-07-22T13:51:48Z",
"State": "Unanswered",
"Answered_by": null,
"Body": "Please turn these off by default. Gamification has no place here. Useless twaddle.",
"Upvotes": 32,
"Labels": [
"Profile"
],
"Participants": 13,
"Answer_count": 13,
"Reply_count": 10,
"Answers": [
{
"IsOffTopic": false,
"Author": "jgmac1106",
"Date": "2022-07-22T16:04:07Z",
"Body": "i agree on off by default. Always default to privacy. I disagree on utility. If the achievements had useful metadata that complied with current industry recommendations the information could be ingested to track role based training requirements of developers, aid in portfolio reviews, and allow users to control their learning data outside of employers. Granted the achievements (little disappointing) are just images for now, but it could be easily extendable to allow parsing, ignestiong, and recording in an immutable ledger.",
"Upvotes": 3,
"Accepted": false,
"Reply_count": 2,
"Replies": [
{
"Author": "SteveALee",
"Body": "That's an interesting idea but a big \"if\" to get the badges representing meaningful development metrics rather than feel good.",
"Date": "2022-07-22T16:18:47Z",
"IsAuthor": true,
"Sentiment": "positive"
},
{
"Author": "seek-dev",
"Body": "Gamifying something just to sell more metadata to corporations who parasitize human privacy is inherently exploitation. The purpose is to manifest addictive behaviour with immaterial rewards. Which is a manipulative function of industrial psychology, though common practice in today's ecosystems of consumer spyware and vacuous social media.",
"Date": "{{datetime}}",
"IsAuthor": false,
"Sentiment": "negative"
}
],
"Sentiment": "negative"
},
{
"IsOffTopic": false,
"Author": "MrSarno",
"Date": "2022-07-22T16:50:03Z",
"Body": "I'm not sure whether you're aware, but there is a setting to disable them here. I don't feel strongly about your suggestion one way or the other. I think people are more likely to check for settings to disable features they dislike than they are to search for features they might hypothetially like and wish to enable. In addition, some people may not think to look for settings, and so there would likely be a significant number of people reporting the lack of achievements as a bug, and / or creating discussions to ask why they don't appear to be working. At least the setting's there for those who want it.",
"Upvotes": 13,
"Accepted": false,
"Reply_count": 2,
"Replies": [
{
"Author": "MlgmXyysd",
"Body": "I agree with this point",
"Date": "2022-07-24T17:37:27Z",
"IsAuthor": false,
"Sentiment": "neutral"
},
{
"Author": "mark-i-m",
"Body": "Thanks for this! I would never have found that setting on my own.",
"Date": "2022-07-25T17:49:57Z",
"IsAuthor": false,
"Sentiment": "neutral"
}
],
"Sentiment": "neutral"
}
This is the code i wrote
import json
file_json = open("/content/drive/MyDrive/Lab_SC/gh_discussions_badges.json")
data = json.load(file_json)
print (data)
This is the error
JSONDecodeError: Extra data: line 855 column 1 (char 26418)
CodePudding user response：
Your data are JSON objects separated by a new line. We need to turn into a JSON array. Maybe this will work?
import json
with open("/content/drive/MyDrive/Lab_SC/gh_discussions_badges.json") as f:
text = f.read()
# Find all line break, where first line is title
text = text.replace('}\n{\n"Title"','},\n{\n"Title"')
# Wrapping in array
json_text = '[' text ']'
data = json.loads(json_text)
print(data)
CodePudding user response：
Your file is not JSON.
You're essentially doing the following:
import json
json.loads("{}{}")
Which gives
JSONDecodeError: Extra data: line 1 column 3 (char 2)
Either you need an array of objects [{},{}] or a single object {}.