Is there a way find download/popularity stats for depending packages?-CodePudding

For example I have a driver for elasticsearch and list of modules, which are build with using of this driver. It's about ~800 modules.

Is there a way to sort this list by popularity or download, so I can find the most used dependent package? I guess it will help to find useful libraries or ODM based on native driver.

I have found several community npm-based stats, but I'll never seen anything like it.

CodePudding user response：

You can run some script like this. It is basically fetching all the dependents, then for each dependent, acquiring the score.

const pkg = "@elastic/elasticsearch";
const dependents = [];

const fetchAll = async (offset = 0) => {
  const res = await fetch(
    `https://www.npmjs.com/browse/depended/${pkg}?offset=${offset}`,
    { headers: { "x-spiferack": "1" }, method: "GET" }
  );
  const data = await res.json();
  dependents.push(...data.packages);
  if (data.hasNext) await fetchAll(offset   data.paginationSize);
};

const main = async () => {
  await fetchAll();
  const result = await Promise.all(
    dependents.map(async ({ name }) => {
      const res = await fetch(
        `https://registry.npmjs.com/-/v1/search?text=${name}&size=1`
      );
      const data = await res.json();
      return { name, score: data.objects[0]?.score };
    })
  );

  // console.log(JSON.stringify(result));
  // TODO: sort then display your result here
};

main();

The result will look something like this before sorting:

[
  {
    "name": "winston-elasticsearch",
    "score": {
      "final": 0.3617392174769229,
      "detail": {
        "quality": 0.5468432882273897,
        "popularity": 0.23232512016539603,
        "maintenance": 0.3324926827166212
      }
    }
  },
  {
    "name": "mongoosastic",
    "score": {
      "final": 0.3562854605860561,
      "detail": {
        "quality": 0.5583413799576047,
        "popularity": 0.2060467998060229,
        "maintenance": 0.3333333333333333
      }
    }
  },
  // ...
]

Note that this is a hack (using NPM's internal APIs and excessively searching the registry), and NPM will likely throw some error like too many requests or connection timeout, especially if your number of packages is too much, and you are not cooling down (you can implement it here, either by changing the logic where I have used Promise.all or using got which already has such features).

Moreover, these API endpoints don't appear to be consistent enough (like the dependents one was giving only ~400 entries for @elastic/elasticsearch while on the site it shows ~800, and the score one was not giving score for certain packages though it should). It will be better for you to use some service like npms.io that properly validates (and caches) the results. The approach will be similar, and pretty much straightforward to implement.

There are also certain packages on NPM that already do/used to do this. Look them up here to see if any is working properly.

Instead of the /-/v1/search hack, you can also fetch https://www.npmjs.com/package/${name} with "x-spiferack": "1" header. But it will only give you download count.