Do a crawler python, screening of saving data,

The data obtained from the mysql database list,

List1

[{' title ':' I am heading 1 ', 'url' : 'I am site 1}, {' title' : 'I am heading 2', 'url' : 'I am a site 2'}, "title" : 'I am heading 3', 'url' : 'I am site 3}]

The latest data from the mining:

List2

[{' title ':' I am the latest 1 ', 'url' : 'I am a new web site 1}, {" title ":' I am the latest 2 ', 'url' : 'I am a new site 2}," title ":' I am heading 1 ', 'url' : '1' I'm a url}]


Excuse me, how to delete list2 neutralization list1 same data? And get a new list data
List3

[{' title ':' I am the latest 1 ', 'url' : 'I am a new web site 1}, {" title ":' I am the latest 2 ', 'url' : 'I am a new site 2}]

CodePudding user response:

Feel can all inserted into the database directly, in the database, the title and the url is set to the primary key, each time you insert using the replace into tablename XXX this statement, it means that every time insert for the database to have the same data, have update, no insert

CodePudding user response:

 list1=[{' sf, ':' DFS '}, {' SFS ':' DFDS}] 
List2=[{' sf, ':' DFS '}, {' sf0s' : 'DFDS'}, {' key ':' SFSD}] 
List3=[_2 for _2 list2 if _2 in the not in list1] 
Print (list3) 

[{' sf0s' : 'DFDS'}, {' key ':' SFSD}]