Home > front end >  How to fetch data from DOM, process it and then send it back in Chrome Extension MV3
How to fetch data from DOM, process it and then send it back in Chrome Extension MV3

Time:03-01

I don't know what's the best way to phrase it, but I decided to make a post here since I've been searching for a few days and couldn't find an exact answer to what I want to do.

So, basically I'm trying to develop a simple Chrome extension to check if users on Reddit are bots or not. The extension would:

  • Fetch all users from the current thread using document.querySelectorAll;
  • Fetch user personal data from reddit;
  • Check which users are bots;
  • Append a [BOT] tag to all bots identified.

At first I tried to do something like this in my background.js:

chrome.tabs.onUpdated.addListener((tabId, changeInfo, tab) => {
    if (changeInfo.status === 'complete'){

        chrome.scripting.executeScript({
            target: { tabId: tabId},
            func: getAuthors,
        },
        (authors) =>{
            authors.forEach(element => {
                getAbout(element).then(about =>{
                    getComments(element).then(comments =>{
                        isBot(about,comments).then(response =>{   
                           //ADD TAG
                        });
                    });
                });
            });
        })
    }
});

The function getAuthors is able to sucessfully access the document and retrieve the data, the problem happens when I have to access it again to append the tags.

Storing an array of authors and passing it in another chrome.scripting.executeScript doesn't work, since the execution won't wait for the first chrome.scripting.executeScript to finish and will execute it immediately.

I've tried inverting the logic and use messaging as well, this time in my contentScript.js:

chrome.runtime.sendMessage('get-names', (names) => {
   console.log('received user data', names);

   names.forEach(element => {
      element.parentElement.appendChild(tag);
   });
});

Now I have a way to append the tags, but I can't send the data to process it in the first place, since, as far as I know, you can only send a payload on response. if I send a message from my background.js instead, I end up falling in a similar situation with the scripting and its asynchronous execution.

Is there an easy way to do this?

CodePudding user response:

While it is possible to implement this task using a background script, a simpler and better solution is to do everything in the content script.

The content script can make changes to DOM immediately and it can make a network request to the current site's URL to fetch user's details. The network request will be asynchronous though, otherwise it would introduce unpleasant delays in page loading.

Here's the overall scheme:

  1. The content script loads at document_start when DOM is empty.
  2. It reads the previously stored data about the users it already checked in the past. The best storage here would be the site's own window.localStorage because it's synchronous and belongs to the same internal tab process so it's fast to read.
  3. Use MutationObserver to detect the <a> elements added while the page loads as well as afterwards to support the new reddit UI (it constructs the page dynamically) and various userscripts/extensions that automatically expand the threads in the old reddit UI.
  4. For each matching element get the cached data.
  5. If absent, make a network request, apply the result, store it in the storage.

manifest.json:

{
  "content_scripts": [{
    "matches": ["*://www.reddit.com/*", "*://old.reddit.com/*"],
    "js": ["content.js"],
    "css": ["content.css"],
    "run_at": "document_start"
  }],

content.css:

a.is-bot::after {
  content: '[BOT]';
  color: red;
  margin-left: .25em;
}

content.js:

const CLASS = 'is-bot';
const LS_KEY = 'b:';
const USER_URL = location.origin   '/user/';
const SEL = `a[href^="/user/"], a[href^="${USER_URL}"]`;
const checkedElems = new WeakSet();
const promises = {};
new MutationObserver(onMutation)
  .observe(document, {subtree: true, childList: true});

function onMutation(mutations) {
  for (const {addedNodes} of mutations) {
    for (const n of addedNodes) {
      const toCheck = n.tagName === 'a'
        ? n.href.startsWith(USER_URL) && [n]
        : n.firstElementChild && n.querySelectorAll(SEL);
      for (const el of toCheck || []) {
        if (checkedElems.has(el)) continue;
        checkedElems.add(el);
        const name = el.href.slice(USER_URL.length);
        const isBot = localStorage[LS_KEY   name];
        if (isBot === '1') {
          el.classList.add(CLASS);
        } else if (isBot !== '0') {
          (promises[name] || (promises[name] = checkUser(el, name)))
            .then(res => res && el.classList.add(CLASS));
        }
      }
    }
  }
}

async function checkUser(el, name) {
  const isBot = await (await fetch(API_URL   name)).json();
  localStorage[LS_KEY   name] = isBot ? '1' : '0';
  delete promises[name];
  return isBot;
}
  • Related