I am using the Laravel Goutte package to perform some webscraping - the following code works and returns a lot of data, I am trying to filter out only the bit of data I require.
If I load up the browser (whilst injecting jQuery into the page) I am able to get the data I need using jQuery using the following in the console jQuery('ea-proclub-overview')[0]; - I am basically trying to do the equivalent of this command within the Laravel/Goutte instance below.
Using jQuery('ea-proclub-overview')[0].customCrestBaseUrl; in the console I get the exact URL I need - https://fifa21.content.easports.com/fifa/fltOnlineAssets/05772199-716f-417d-9fe0-988fa9899c4d/2021/fifaweb/crests/256x256/l'
Below is my PHP code - I am getting back in the $node variable but I am unsure how to only return the customCrestBaseUrl so it gives me the URL.
$client = new Client();
$client->setServerParameter('HTTP_USER_AGENT', 'Mozilla/5.0 (X11; Linux i686; rv:78.0) Gecko/20100101 Firefox/78.0');
$client->setServerParameter('REFERER', 'https://www.ea.com/');
$url = 'https://www.ea.com/en-gb/games/fifa/pro-clubs/ps5-xbsxs/overview?clubId=2552&platform=ps5';
$crawler = $client->request('GET', $url);
$crawler->filter('ea-proclub-overview')->each(function ($node) {
dd($node);
});
Expected Result
<ea-proclub-overview endpoints="{"settingsEndpoint":"https://proclubs.ea.com/api/fifa/settings","seasonalStatsEndpoint":"https://proclubs.ea.com/api/fifa/clubs/seasonalStats","clubsInfoEndpoint":"https://proclubs.ea.com/api/fifa/clubs/info","matchesEndpoint":"https://proclubs.ea.com/api/fifa/clubs/matches","memberStatEndpoint":"https://proclubs.ea.com/api/fifa/members/stats","memberCareerStatEndpoint":"https://proclubs.ea.com/api/fifa/members/career/stats"}" colors="{"currentDivision":{"startColor":"#FA4358","endColor":"#FA4358"},"nextDivision":{"relegationColor":"#FA4358","pointsColor":"#FA4358"},"pieChart":{"winsColor":"#19A863","lossesColor":"#C4010D","tiesColor":"#282D3B"},"stats":{"wins":{"startColor":"#19A863","endColor":"#94D85D"},"losses":{"startColor":"#C4010D","endColor":"#F80245"},"ties":{"startColor":"#282D3B","endColor":"#282D3B"}}}" match-type="["gameType9","gameType13"]" headers-labels="{"points":"Points","stats":{"wins":{"label":"Wins","description":"Wins"},"losses":{"label":"Losses","description":"Losses"},"ties":{"label":"Draws","description":"Draws"}}}" division-labels="{"title":"Division Ranking","currentDivisionTitle":"Current Division","nextDivisionTitle":"Points To Next Division","seasons":"Season","record":"Record","points":"Points","gamesPlayed":"Games Played","gamesRemaining":"Games Remaining","divisionImgBaseUrl":"https://media.contentapi.ea.com/content/dam/eacom/fifa/pro-clubs/divisioncrest","stats":{"wins":"W","losses":"L","ties":"D"}}" progressbar-labels="{"div":"Div","promotion":"Promotion","relegation":"Relegation","title":"Title"}" members-labels="{"title":"Members","linkText":"View All Members","linkUrl":"members","totalTitle":"Total Members","totalCountsLabel":"Total","memberDetails":{"proOverall":"Overall Rating","ratingAve":"Average Match Rating","gamesPlayed":"Games Played"},"memberPosition":{"defender":"Defender","forward":"Forward","goalkeeper":"Goalkeeper","midfielder":"Midfielder"},"positions":{"defender":"Defenders","forward":"Forwards","goalkeeper":"Goalkeepers","midfielder":"Midfielders"},"defaultMemberAvatar":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/avatar.png"}" match-labels="{"title":"Last Match","linkText":"View All Match History","linkUrl":"match-history","altTitle":"No match data was found"}" trophies-labels="{"title":"Trophies","cupsLabel":{"leaguesWon":"Leagues Won","titlesWon":"Titles Won","totalCupsWon":"Total Cups Won"},"cupsImg":{"leaguesWonImgUrl":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/league-titles-21.png","titlesWonImgUrl":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/all-tiles-21.png","totalCupsWonImgUrl":"https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/cups-won-21.png"}}" history-labels="{"title":"Club History","subTitle":"Overall Record","pts":"Points","division":"Division","historyDetails":{"seasons":"Seasons Played","totalGames":"Total Games","titlesWon":"Titles Won","bestPoints":"Highest Points Total","promotions":"Promotions","relegations":"Relegations"},"stats":{"wins":"Wins","losses":"Losses","ties":"Draws"},"statsShort":{"wins":"W","losses":"L","ties":"D"},"progressBar":{"title":"Best Season Finish","tipLabel":"DIV","startColor":"#9B7801","endColor":"#F9F1A5","divisionBaseUrl":"https://media.contentapi.ea.com/content/dam/eacom/fifa/pro-clubs/divisioncrest"}}" translations="{"4543827":"East Coast US","5723475":"West Coast US","5719381":"Western Europe","4539733":"Eastern Europe","5129557":"Northern Europe","5457237":"Southern Europe","4344147":"British Isles","5456205":"South America","4407629":"Central America","4281153":"Asia","4281683":"Australia / New Zealand"}" crest-base-url="https://fifa21.content.easports.com/fifa/fltOnlineAssets/05772199-716f-417d-9fe0-988fa9899c4d/2021/fifaweb/crests/256x256/l" custom-crest-base-url="https://fifa21.content.easports.com/fifa/fltOnlineAssets/05772199-716f-417d-9fe0-988fa9899c4d/2021/fifaweb/crests/256x256/l" default-crest-url="https://media.contentapi.ea.com/content/dam/ea/fifa/fifa-21/pro-clubs/common/pro-clubs/crest-default.png" loading-image="https://media.contentapi.ea.com/content/dam/eacom/fifa/pro-clubs/loading-animation.png" default-club-name="Disbanded"></ea-proclub-overview>
Actual Result
-- Too much post but it all of the HTML & XML
Below is pastebin of the entire response from the $crawler when dumped out using dd(). https://pastebin.com/qxUTpu9p
CodePudding user response：
According to the documentation:
$customCrestBaseUrl = $crawler
->filter('ea-proclub-overview')
->first()
->extract(['custom-crest-base-url'])
;