Do the crawler climb took some questions and answers in the mongo database data storage, and my single dictionary design is: {" k1 ": the title of" q&a ", "url" : "document links", "k2" : "the q&a time", "k3" : "to answer a few," {" zhang ", "his answer is... ", "bill" : "his answer is... ", "detective" : "his answer is... "}}, my data structure is that the only won't change is that k1 and url, is the question and answer the questions and answers document link address will not change, the remaining anyone at any time to review, so this dictionary will be regularly updated, such specific what to do, how data storage operation, how to operate the query

CodePudding user response:

I saw the demand, probably think should meet the requirements, for reference:


Two table
Issue information table
Question_tbl (
Qid automatically increase field
K1
Url
)
Answer information table
Anwser_tbl (
An automatic increase field
Qid ID list
Anwser_name answer people
Anwser_comment answer content
Anwser_time response time
)


Question_tbl and anwser_tbl qid associations,
Based upon the qid anwser_tbl grouping statistics k2 (last answer time), k3 (quantity)

CodePudding user response:

reference 1/f, old coconut response:

I saw the demand, probably think should meet the requirements, for reference:


Two table
Issue information table
Question_tbl (
Qid automatically increase field
K1
Url
)
Answer information table
Anwser_tbl (
An automatic increase field
Qid ID list
Anwser_name answer people
Anwser_comment answer content
Anwser_time response time
)


Question_tbl and anwser_tbl qid associations,
Based upon the qid anwser_tbl grouping statistics k2 (last answer time), k3 (quantity)

You in this line of thought, I see, really can do it, but actually I this data volume is too big, a lot of comments, 70-80, the data redundancy is too big, q&a entry also has hundreds of thousands of article, said online q&a type data with mongo storage is the most appropriate, but I'm worried about is the first time to take no problem, repeat crawl can create data redundancy, when in fact question and answer the topic of weight is one of the biggest, will never change, there will be other data update dictionary continue to insert it, just like a simple version of the blog comment system of storage, but I have no idea to operate, don't know where to start

CodePudding user response:

Actually the previous design, do not have what problem, is also a common design, redundancy is not large, or does not have what redundancy,

Instead, use your own table design, crawl after the first time, every time saved, will be more trouble:
Because for every answer of the contents of the list to determine whether there is a repeated
And also not rigorous, one thousand have 2 people to return to the same content, such as "good", you don't know is many times repeat lead to crawl, or single crawl normal content but just the same reply

Suggestion:
with the design of others
In addition, if you don't want to use their own, is not can't,

Here is helping to help you optimize your single table fields and structure

 
{
The topic of "question_title" : "the question," 
"Question_url" : "the problem of link", 
"Qestion_created_time" : "the q&a time," 
"Answers" : {
"Total_num" : "answer a few," 
"A list" : [
{
"Zhang SAN" : "his answer is... "
"Bill" : "his answer is... "
"Fifty" : "his answer is... "
} 
] 
} 
}