Python for loop question, alternative solutions-CodePudding

I'm working on an application about fetching datas from imdb. I just wrote those codes :

import time
from bs4 import BeautifulSoup as bs
import requests

source = requests.get("https://www.imdb.com/list/ls070233852/")
soup = bs(source.content,'html.parser')
movies = soup.find_all("div",attrs={"class":"mode-detail"})
basliklar = []
yillar = []
for baslik in movies:
    urunDivs = baslik.find("div",attrs={"class":"lister-item-content"})
    for i in urunDivs:
        tumTextler = urunDivs.find("h3",attrs={"class":"lister-item-header"})
        for x in tumTextler:
            yilString = tumTextler.find("span",attrs={"class":"lister-item-year"}).text
            yillar.append(yilString)
            baslikString = urunDivs.find("a").text
            if baslikString not in basliklar:
                basliklar.append(baslikString) 
                 
for i in yillar:
    print(i)

As you can think, a lot of movies made in same year. So if i use

if yilString not in yillar:
    yillar.append(yilString)

It deletes a lot of same years. But if i don't use those codes, it prints a lot of same year same time for for loop. Is there any alternative solution about this problem? I'm stuck.

CodePudding user response：

I don't quite understand what you want to achieve with your code. Here is a simple example to get basic movie data. We use a unique value as the movie ID. I recommend using the API. 500 calls per month are free. You can also import the library - IMDbPY and use Cinemagoer

from bs4 import BeautifulSoup as bs
import requests

source = requests.get("https://www.imdb.com/list/ls070233852/")
soup = bs(source.content, 'html.parser')
movies = []
for baslik in soup.find_all("div", class_="lister-item-content"):
    movie = {
        'movie_id': baslik.find("h3", class_="lister-item-header").find('a').get('href').split('/')[2],
        'movie_title': baslik.find("h3", class_="lister-item-header").find('a').getText(strip=True),
        'movie_year': ''.join(c for c in baslik.find("span", class_="lister-item-year").getText(strip=True) if c.isdigit()),
        'movie_rating': baslik.find("span", class_="ipl-rating-star__rating").getText(strip=True),
        'movie_genres': baslik.find("span", class_="genre").getText(strip=True),
        'movie_runtime': baslik.find("span", class_="runtime").getText(strip=True),
        'movie_description': baslik.find("p").findNext("p").getText(strip=True)
    }
    movies.append(movie)
    print(movie)

OUTPUT:

{'movie_id': 'tt0096913', 'movie_title': 'Best of the Best', 'movie_year': '1989', 'movie_rating': '6.4', 'movie_genres': 'Action, Drama, Sport', 'movie_runtime': '97 min', 'movie_description': 'A team from the United States is going to compete against Korea in a Tae Kwon Do tournament. The team consists of fighters from all over the country - can they overcome their rivalry and work together to win?'}
{'movie_id': 'tt0095016', 'movie_title': 'Die Hard', 'movie_year': '1988', 'movie_rating': '8.2', 'movie_genres': 'Action, Thriller', 'movie_runtime': '132 min', 'movie_description': 'An NYPD officer tries to save his wife and several others taken hostage by German terrorists during a Christmas party at the Nakatomi Plaza in Los Angeles.'}
{'movie_id': 'tt0082971', 'movie_title': 'Raiders of the Lost Ark', 'movie_year': '1981', 'movie_rating': '8.4', 'movie_genres': 'Action, Adventure', 'movie_runtime': '115 min', 'movie_description': "In 1936, archaeologist and adventurer Indiana Jones is hired by the U.S. government to find the Ark of the Covenant beforeAdolf Hitler's Nazis can obtain its awesome powers."}
{'movie_id': 'tt0111503', 'movie_title': 'True Lies', 'movie_year': '1994', 'movie_rating': '7.3', 'movie_genres': 'Action, Comedy, Thriller', 'movie_runtime': '141 min', 'movie_description': 'A fearless, globe-trotting, terrorist-battling secret agent has his life turned upside down when he discovers his wife might be having an affair with a used-car salesman while terrorists smuggle nuclear war heads into the United States.'}
{'movie_id': 'tt0325710', 'movie_title': 'The Last Samurai', 'movie_year': '2003', 'movie_rating': '7.8', 'movie_genres': 'Action, Drama', 'movie_runtime': '154 min', 'movie_description': 'An American military advisor embraces the Samurai culture he was hired to destroy after he is captured in battle.'}
{'movie_id': 'tt0112864', 'movie_title': 'Die Hard: With a Vengeance', 'movie_year': '1995', 'movie_rating': '7.6', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '128 min', 'movie_description': 'John McClane and a Harlem store owner are targeted by German terrorist Simon in New York City, where he plans to rob the Federal Reserve Building.'}
{'movie_id': 'tt0096256', 'movie_title': 'They Live', 'movie_year': '1988', 'movie_rating': '7.2', 'movie_genres': 'Action, Horror, Sci-Fi', 'movie_runtime': '94 min', 'movie_description': 'They influence our decisions without us knowing it. They numb our senses without us feeling it. They control our lives without us realizing it. They live.'}
{'movie_id': 'tt0120815', 'movie_title': 'Saving Private Ryan', 'movie_year': '1998', 'movie_rating': '8.6', 'movie_genres': 'Drama, War', 'movie_runtime': '169 min', 'movie_description': 'Following the Normandy Landings, a group of U.S. soldiers go behind enemy lines to retrieve a paratrooper whose brothers have been killed in action.'}
{'movie_id': 'tt0111257', 'movie_title': 'Speed', 'movie_year': '1994', 'movie_rating': '7.3', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '116 min', 'movie_description': 'A young police officer must prevent a bomb exploding aboard a city bus by keeping its speed above 50 mph.'}
{'movie_id': 'tt0113277', 'movie_title': 'Heat', 'movie_year': '1995', 'movie_rating': '8.3', 'movie_genres': 'Action, Crime, Drama', 'movie_runtime': '170 min', 'movie_description': 'A group of high-end professional thieves start to feel the heat from the LAPD when they unknowingly leave a clue at their latest heist.'}
{'movie_id': 'tt0122690', 'movie_title': 'Ronin', 'movie_year': '1998', 'movie_rating': '7.2', 'movie_genres': 'Action, Crime, Thriller', 'movie_runtime': '122 min', 'movie_description': 'A freelancing former U.S. Intelligence Agent tries to track down a mysterious package that is wanted by the Irish and the Russians.'}
{'movie_id': 'tt0108171', 'movie_title': 'Sniper', 'movie_year': '1993', 'movie_rating': '6.1', 'movie_genres': 'Action, Thriller, War', 'movie_runtime': '98 min', 'movie_description': 'A veteran US Marine sniper is partnered with a rookie sniper as his spotter to take out a politician and a rebel leader in the jungles of Panama.'}
{'movie_id': 'tt0440963', 'movie_title': 'The Bourne Ultimatum', 'movie_year': '2007', 'movie_rating': '8', 'movie_genres': 'Action, Mystery, Thriller', 'movie_runtime': '115 min', 'movie_description': 'Jason Bourne dodges a ruthless C.I.A. official and his Agents from a new assassination program while searching for the origins of his life as a trained killer.'}
{'movie_id': 'tt0088247', 'movie_title': 'The Terminator', 'movie_year': '1984', 'movie_rating': '8.1', 'movie_genres': 'Action, Sci-Fi', 'movie_runtime': '107 min', 'movie_description': "A human soldier is sent from 2029 to 1984 to stop an almost indestructible cyborg killing machine, sent from the same year, which has been programmed to execute a young woman whose unborn son is the key to humanity's future salvation."}
{'movie_id': 'tt0936501', 'movie_title': 'Taken', 'movie_year': '2008', 'movie_rating': '7.8', 'movie_genres': 'Action, Crime, Thriller', 'movie_runtime': '90 min', 'movie_description': 'A retired CIA agent travels across Europe and relies on his old skills to save his estranged daughter, who has been kidnapped while on a trip to Paris.'}
{'movie_id': 'tt2302755', 'movie_title': 'Olympus Has Fallen', 'movie_year': '2013', 'movie_rating': '6.5', 'movie_genres': 'Action, Thriller', 'movie_runtime': '119 min', 'movie_description': 'Secret Service agent Mike Banning finds himself trapped inside the White House in the wake of a terrorist attack and works with national security to rescue the President from his kidnappers.'}
{'movie_id': 'tt0102614', 'movie_title': 'Out for Justice', 'movie_year': '1991', 'movie_rating': '6.1', 'movie_genres': 'Action, Crime, Drama', 'movie_runtime': '91 min', 'movie_description': "The gruesome murder of a Brooklyn Detective will turn the case into a personal vendetta when the deceased's best friend and fellow officer will unleash an all-out attack against a psychotic Mafia enforcer's brutal gang."}
{'movie_id': 'tt0102685', 'movie_title': 'Point Break', 'movie_year': '1991', 'movie_rating': '7.2', 'movie_genres': 'Action, Crime, Thriller', 'movie_runtime': '122 min', 'movie_description': 'An F.B.I. Agent goes undercover to catch a gang of surfers who may be bank robbers.'}
{'movie_id': 'tt0111825', 'movie_title': 'Zero Tolerance', 'movie_year': '1994', 'movie_rating': '5.3', 'movie_genres': 'Action, Crime, Drama', 'movie_runtime': '88 min', 'movie_description': "Jeff is an FBI agent sent to pick up Ray Manta, a member of the White Hand drug cartel, from a Mexican jail. Manta escapes, and gets revenge by killing Jeff's family. Kowalski, another ...See full summary»"}
{'movie_id': 'tt0103064', 'movie_title': 'Terminator 2: Judgment Day', 'movie_year': '1991', 'movie_rating': '8.6', 'movie_genres': 'Action, Sci-Fi', 'movie_runtime': '137 min', 'movie_description': 'A cyborg, identical to the one who failed to kill Sarah Connor, must now protect her ten-year-old son John from a more advanced and powerful cyborg.'}
{'movie_id': 'tt0090605', 'movie_title': 'Aliens', 'movie_year': '1986', 'movie_rating': '8.4', 'movie_genres': 'Action, Adventure, Sci-Fi', 'movie_runtime': '137 min', 'movie_description': 'Fifty-seven years after surviving an apocalyptic attack aboard her space vessel by merciless space creatures, Officer Ripley awakens from hyper-sleep and tries to warn anyone who will listen about the predators.'}
{'movie_id': 'tt0088944', 'movie_title': 'Commando', 'movie_year': '1985', 'movie_rating': '6.7', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '90 min', 'movie_description': 'A retired Special Forces colonel tries to save his daughter, who was abducted by his former subordinate.'}
{'movie_id': 'tt0056172', 'movie_title': 'Lawrence of Arabia', 'movie_year': '1962', 'movie_rating': '8.3', 'movie_genres': 'Adventure, Biography, Drama', 'movie_runtime': '218 min', 'movie_description': 'The story ofT.E. Lawrence, the English officer who successfully united and led the diverse, often warring, Arab tribes during World War I in order to fight the Turks.'}
{'movie_id': 'tt0107027', 'movie_title': 'Godzilla vs. Mechagodzilla II', 'movie_year': '1993', 'movie_rating': '6.5', 'movie_genres': 'Action, Adventure, Drama', 'movie_runtime': '108 min', 'movie_description': 'The United Nations assembles the ultimate weapon to defeat Godzilla, while scientists discover a fresh pteranodon egg on a remote Japanese island.'}
{'movie_id': 'tt0120657', 'movie_title': 'The 13th Warrior', 'movie_year': '1999', 'movie_rating': '6.6', 'movie_genres': 'Action, Adventure, History', 'movie_runtime': '102 min', 'movie_description': 'A man, having fallen in love with the wrong woman, is sent by the sultan himself on a diplomatic mission to a distant land as an ambassador. Stopping at a Viking village port to restock on supplies, he finds himself unwittingly embroiled in a quest to banish a mysterious threat in a distant Viking land.'}
{'movie_id': 'tt0187738', 'movie_title': 'Blade II', 'movie_year': '2002', 'movie_rating': '6.7', 'movie_genres': 'Action, Horror, Sci-Fi', 'movie_runtime': '117 min', 'movie_description': 'Blade forms an uneasy alliance with the vampire council in order to combat the Reapers, who are feeding on vampires.'}
{'movie_id': 'tt0083944', 'movie_title': 'First Blood', 'movie_year': '1982', 'movie_rating': '7.7', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '93 min', 'movie_description': 'A veteran Green Beret is forced by a cruel Sheriff and his deputies to flee into the mountains and wage an escalating one-man war against his pursuers.'}
{'movie_id': 'tt2103281', 'movie_title': 'Dawn of the Planet of the Apes', 'movie_year': '2014', 'movie_rating': '7.6', 'movie_genres': 'Action, Adventure, Drama', 'movie_runtime': '130 min', 'movie_description': 'The fragile peace between apes and humans is threatened as mistrust and betrayal threaten to plunge both tribes into a war for dominance over the Earth.'}
{'movie_id': 'tt0104684', 'movie_title': 'Hard Boiled', 'movie_year': '1992', 'movie_rating': '7.7', 'movie_genres': 'Action, Crime, Thriller', 'movie_runtime': '128 min', 'movie_description': 'A tough-as-nails cop teams up with an undercover agent to shut down a sinister mobster and his crew.'}
{'movie_id': 'tt0167261', 'movie_title': 'The Lord of the Rings: The Two Towers', 'movie_year': '2002', 'movie_rating': '8.8', 'movie_genres': 'Action, Adventure, Drama', 'movie_runtime': '179 min', 'movie_description': "While Frodo and Sam edge closer to Mordor with the help of the shifty Gollum, the divided fellowship makes a stand against Sauron's new ally, Saruman, and his hordes of Isengard."}
{'movie_id': 'tt0093773', 'movie_title': 'Predator', 'movie_year': '1987', 'movie_rating': '7.8', 'movie_genres': 'Action, Adventure, Horror', 'movie_runtime': '107 min', 'movie_description': 'A team of commandos on a mission in a Central American jungle find themselves hunted by an extraterrestrial warrior.'}
{'movie_id': 'tt0057076', 'movie_title': 'From Russia with Love', 'movie_year': '1963', 'movie_rating': '7.4', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '115 min', 'movie_description': 'James Bond willingly falls into an assassination plot involving a naive Russian beauty in order to retrieve a Soviet encryption device that was stolen by S.P.E.C.T.R.E.'}
{'movie_id': 'tt0065938', 'movie_title': "Kelly's Heroes", 'movie_year': '1970', 'movie_rating': '7.6', 'movie_genres': 'Adventure, Comedy, War', 'movie_runtime': '144 min', 'movie_description': 'A group of U.S. soldiers sneaks across enemy lines to get their hands on a secret stash of Nazi treasure.'}
{'movie_id': 'tt0093428', 'movie_title': 'The Living Daylights', 'movie_year': '1987', 'movie_rating': '6.7', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '130 min', 'movie_description': 'James Bond is sent to investigate a KGB policy to kill all enemy spies and uncovers an arms deal that potentially has major global ramifications.'}
{'movie_id': 'tt0078492', 'movie_title': 'The Wild Geese', 'movie_year': '1978', 'movie_rating': '6.8', 'movie_genres': 'Action, Adventure, Drama', 'movie_runtime': '134 min', 'movie_description': 'A British banker hires a group of British mercenaries to rescue a deposed African President from the hands of a corrupt African dictator.'}
{'movie_id': 'tt0054047', 'movie_title': 'The Magnificent Seven', 'movie_year': '1960', 'movie_rating': '7.7', 'movie_genres': 'Action, Adventure, Western', 'movie_runtime': '128 min', 'movie_description': 'Seven gunfighters are hired by Mexican peasants to liberate their village from oppressive bandits.'}
{'movie_id': 'tt0113189', 'movie_title': 'GoldenEye', 'movie_year': '1995', 'movie_rating': '7.2', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '130 min', 'movie_description': 'Years after a friend and fellow 00 agent is killed on a joint mission, a Russian crime syndicate steals a secret space-based weapons program known as "GoldenEye" and James Bond has to stop them from using it.'}
{'movie_id': 'tt0381061', 'movie_title': 'Casino Royale', 'movie_year': '2006', 'movie_rating': '8', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '144 min', 'movie_description': 'After earning 00 status and a licence to kill, secret agent James Bond sets out on his first mission as 007. Bond must defeat a private banker funding terrorists in a high-stakes game of poker at Casino Royale, Montenegro.'}
{'movie_id': 'tt0068767', 'movie_title': 'Fist of Fury', 'movie_year': '1972', 'movie_rating': '7.2', 'movie_genres': 'Action, Drama, Romance', 'movie_runtime': '106 min', 'movie_description': 'A young man seeks vengence for the death of his teacher.'}
...
{'movie_id': 'tt0116213', 'movie_title': 'Eraser', 'movie_year': '1996', 'movie_rating': '6.1', 'movie_genres': 'Action, Crime, Thriller', 'movie_runtime': '115 min', 'movie_description': 'A Witness Protection specialist becomes suspicious of his co-workers when dealing with a case involving high-tech weapons.'}
{'movie_id': 'tt0181689', 'movie_title': 'Minority Report', 'movie_year': '2002', 'movie_rating': '7.7', 'movie_genres': 'Action, Crime, Mystery', 'movie_runtime': '145 min', 'movie_description': 'In a future where a special police unit is able to arrest murderers before they commit their crimes, an officer from that unit is himself accused of a future murder.'}
{'movie_id': 'tt0120611', 'movie_title': 'Blade', 'movie_year': '1998', 'movie_rating': '7.1', 'movie_genres': 'Action, Horror, Sci-Fi', 'movie_runtime': '120 min', 'movie_description': 'A half-vampire, half-mortal man becomes a protector of the mortal race, while slaying evil vampires.'}
{'movie_id': 'tt0116253', 'movie_title': 'Executive Decision', 'movie_year': '1996', 'movie_rating': '6.5', 'movie_genres': 'Action, Adventure, Thriller', 'movie_runtime': '133 min', 'movie_description': 'When terrorists seize control of an airliner, an intelligence analyst accompanies a commando unit for a midair boarding operation.'}
{'movie_id': 'tt0112851', 'movie_title': 'Desperado', 'movie_year': '1995', 'movie_rating': '7.1', 'movie_genres': 'Action, Crime, Thriller', 'movie_runtime': '104 min', 'movie_description': 'Former musician and gunslinger El Mariachi arrives at a small Mexican border town after being away for a long time. His past quickly catches up with him and he soon gets entangled with the local drug kingpin Bucho and his gang.'}
{'movie_id': 'tt0106977', 'movie_title': 'The Fugitive', 'movie_year': '1993', 'movie_rating': '7.8', 'movie_genres': 'Action, Crime, Drama', 'movie_runtime': '130 min', 'movie_description': 'Dr. Richard Kimble, unjustly accused of murdering his wife, must find the real killer while being the target of a nationwide manhunt led by a seasoned U.S. Marshal.'}
{'movie_id': 'tt0103285', 'movie_title': 'Once Upon a Time in China', 'movie_year': '1991', 'movie_rating': '7.2', 'movie_genres': 'Action, Adventure, History', 'movie_runtime': '134 min', 'movie_description': "Legendary martial arts hero Wong Fei-Hung fights against foreign forces' plundering of China. When Aunt Yee arrives back from America, Wong Fei-Hung assumes the role of her protector."}
{'movie_id': 'tt0087469', 'movie_title': 'Indiana Jones and the Temple of Doom', 'movie_year': '1984', 'movie_rating': '7.5', 'movie_genres': 'Action, Adventure', 'movie_runtime': '118 min', 'movie_description': 'In 1935, Indiana Jones arrives in India, still part of the British Empire, and is asked to find a mystical stone. He then stumbles upon a secret cult committing enslavement and human sacrifices in the catacombs of an ancient palace.'}