pokemon-showdown/tools/set-import/stats
Kirk Scheibelhut b9e5fcb387
Introduce set importer logic (#5749)
`node tools/set-import [version]` can be run to create a
`@pokemon-showdown/sets` package containing sets from Smogon, usage
stats and third party sources. Some notes:

- The build is set up so that `tools/set-import/importer.ts` is
  compiled to `tools/set-import/importer.js` - creating a
  `.tools-dist/set-import` directory for the sole artifact was
  deemed to be overkill
- The sets package is generated such that it can be used easily on
  Node or in a browser (where the large set JSON can be loaded
  asynchronously)
- Supported tiers/formats are mostly arbitrary - those popular
  enough to have a signficant playerbase or analyses on Smogon have
  been included, but additional formats can be added based on demand
- Some set information is redundant for ease of use by downstream
  users: certain formes are split out and information that could
  theoretically be inferred like level/ability/HP IVs are included
  to simplify parsing logic and make the sets more immediately
  available. This results in what should mostly be negligible
  size overhead.
- In a similar vein, display versions of effect names instead of IDs
  are used (name -> ID is trivial, ID -> name requires data lookup)
- All sets pass validation, provided certain simple transformations
  are applied (eg. reverting `battleOnly` formes like Megas)

This tool has primarily been tested to run on Linux - running on
other platforms is not guaranteed to result in error-free output.
2019-09-10 21:15:05 -04:00

48 lines
1.6 KiB
JavaScript
Executable File

#!/usr/bin/env node
// Determines total battle counts per format since the beginning of stats
// collection in order to determine what the fallback dates for the importer's
// `STATISTICS` map should be.
'use strict';
const smogon = require('smogon');
const importer = require('./importer');
const Dex = require('../../.sim-dist/dex').Dex;
const formats = new Map();
for (let gen = 1; gen <= 7; gen++) {
for (const tier of importer.TIERS) {
const format = Dex.getFormat(`gen${gen}${tier}`);
if (format.exists) {
formats.set(format.id, {});
}
}
}
(async () => {
const index = await importer.fetch(smogon.Statistics.URL);
const begin = new Date('Nov 2014');
const end = new Date(smogon.Statistics.latest(index));
end.setDate(end.getDate() + 1);
for (const d = begin; d <= end; d.setMonth(d.getMonth() + 1)) {
const month = `${d.getMonth() + 1}`.padStart(2, '0');
const date = `${1900 + d.getYear()}-${month}`;
const raw = await importer.fetch(`${smogon.Statistics.URL}${date}/`);
for (const format of formats.keys()) {
// Some formats changed names after Gen 6 (eg. 'ou' -> 'gen6ou.json' etc), but for
// our purposes this doesn't really matter as there should still be the data we need
// under the correct format ID.
if (raw.includes(format)) {
try {
const usage = await importer.fetch(`${smogon.Statistics.URL}${date}/${format}-1500.txt`);
formats.get(format)[date] = Number(usage.match(/Total battles: (.*)/)[1]);
} catch (err) {
if (!err.message.startsWith('HTTP 404')) throw err;
}
}
}
}
console.log(JSON.stringify(Array.from(formats.entries()), null, 2));
})().catch(err => console.error(err));