Leahnaya/Magic-Spoiler
1
0
Fork 0
mirror of https://github.com/Cockatrice/Magic-Spoiler.git synced 2026-04-22 15:07:31 -05:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
226 Commits 3 Branches 10 Tags 76 MiB
075faa71ac
Commit Graph

55 Commits

Author SHA1 Message Date
dev-id
78f5dd2df9 Improved no-cards set debug print 2017-09-08 10:12:15 -05:00
dev-id
fa10639000 Don't make set files for sets without cards. 
Don't put them in spoiler.json
 2017-09-07 11:21:06 -05:00
Dave
1b549638dc Merge pull request #136 from dev-id/small-getimage-fx-refactor 
Refactor get_image_urls function to use only `setinfo` as input.
 2017-09-04 10:46:14 -05:00
Dave
9d00b4f05a Error log to YAML (#137) 
* Error log to YAML

* Also dump debug print to YAML
 2017-09-04 10:45:50 -05:00
dev-id
9693dad628 Refactor get_image_urls function to use only setinfo as input. 2017-09-02 21:21:29 -05:00
Cheldra
d042f71009 New set_info value mythicCode (#135) 
* New set_info value mythicCode

Mythicspoiler is using IXA rather than XLN for some reason. Creating a new parameter to account for this.

* mythicCode

* mythicCode

* Mtgs name fixes

* Mtgs name fixes

* Set mythicCode to code if not present in setinfo
 2017-09-02 10:25:22 -05:00
tooomm
77ec60570e remove old sets and better correction files (#133) 
* remove old sets and better correction files

* debug manual_sets

* Update main.py

* little tweaks
 2017-08-26 06:23:03 +02:00
tritoch
9d5f723fc8 Use scryfall data for C17 (disables all other sources) (#127) 
* Use scryfall data for C17 (disables all other sources)

* Allow individual sets to be scryfall-only sourced.

Remove prototyped variables (and add one...)
 2017-08-12 23:20:24 -05:00
tritoch
a58e8fac01 Exit with failure on XSD invalid 2017-07-10 22:59:27 -05:00
tooomm
88c93c013d adjust print in log 
grammar
 2017-07-10 15:09:13 +02:00
tritoch
529a4da3d9 Remove cards from MTGS RSS that aren't in gallery (#112) 2017-07-07 16:32:02 -05:00
tritoch
f74bd35c8d Eliminate 'split_cards' array. Use 'names' key to determine split cards. (#111) 2017-07-07 16:27:50 -05:00
tritoch
39f6243846 Prettify set xml files (#110) 2017-07-07 16:27:36 -05:00
tritoch
04e6a1892f Verify spoiler.xml against Cockatrice's XSD file (#105) 
Verifies spoiler.xml against Cockatrice's XSD file

verify_xml currently takes an xml file and XSD as a string

Prints a pass/fail above XML dump

* Re-order xml writing to pass XSD

* Improved XSD verification

Now prints error

Now handles malformed XML or XSD
 2017-07-06 21:37:11 -05:00
tritoch
dc9b9b7a48 Refactor set_info, download_images to scraper sub 
Refactor set_info to align with mtgjson keys.

Move download_images to wizards_scraper
 2017-07-06 19:46:26 -05:00
tritoch
5b987d28cf Change input files to YAML (#99) 
Input files to yaml

Deduplicate file verification, move it out to module.

Remove commentjson requirement
 2017-07-06 14:25:10 -05:00
Lee Matos
1dd538d5a1 First pass refactoring scrapers into separate modules (#98) 
Splits off the respective scrapers into submodules (mtgs_scraper.py, scryfall_scraper.py, mythic_scraper.py, wizards_scraper.py)
 2017-07-05 20:44:45 -05:00
tritoch
80c09b1e88 After MTGJSON adds the sets, we don't want them anymore. Goodbye spoiler season! 2017-07-02 11:07:16 -05:00
Lee Matos
91cc9559e5 refactor loading json (#77) 
* refactor loading json
 2017-07-02 10:57:23 -05:00
tooomm
f320660a22 remove split card 2017-07-01 01:32:12 +02:00
tritoch
4e43b90156 Write AllSets.json 
Don't scrape scryfall if we disable comparison

Toggle for Dumping Error log
 2017-06-30 09:42:36 -05:00
tritoch
c32b280391 Pass set name to fullspoil for automatic URL creation 2017-06-27 12:45:42 -05:00
tritoch
fe96df161d Full Spoil preparations 
Handle both WOTC gallery formats
 2017-06-27 12:07:02 -05:00
tritoch
ec7270c524 Log Cleanup 
* Scryfall debug log off by default

* Scryfall logging disabled by default.

Manual corrections print on one line.
 2017-06-23 09:20:40 -05:00
tritoch
1ad19e2d30 Log fix (#49) 
* Don't show error log if none.
 2017-06-22 23:53:58 -05:00
tritoch
60279ab3f9 Error Fix (#43) 
* Errors with loading input files now cause CI to fail

* Misc syntax corrections

* Images for Reason // Believe
 2017-06-22 08:55:27 -05:00
tritoch
6d21eb5ca8 Scrape MTGS for images. Fix WOTC URL (#36) 
* Scrape MTGS for images

* Fix WOTC card gallery URL.

* Additional Aftermath Fixes
 2017-06-21 12:44:39 -05:00
tritoch
eba81f48d7 Add try/except blocks for file loading. (#31) 2017-06-20 14:52:53 -05:00
tritoch
0f2d772821 Return of Corrections (#29) 2017-06-20 12:33:32 -05:00
tritoch
b471f1073d Fix Typing 
Type for printing
 2017-06-20 11:40:25 -05:00
tritoch
3572607c47 Merge branch 'argument-parser' into master 2017-06-20 11:35:25 -05:00
tritoch
5d48d567c3 Add command line argument parser. 
Can change a preset by adding `preset=newvalue` to command line execution.
 2017-06-20 11:33:07 -05:00
tritoch
75421ddd2d Add command line argument parser. 
Can change a preset by adding `preset=newvalue` to command line execution.
 2017-06-20 11:29:03 -05:00
tritoch
c68da7ccab Types and errorlog improvements (#26) 
* Ignore corrected files in Error.
Card Type/Types detection improvement.

* Merge fix

Bad Merge Conflict Resolution
 2017-06-20 09:50:16 -05:00
tritoch
47234b17d2 Corrections Post Scrape, Unicode JSON 
Corrections should be after all scraping. Output files should be Unicode.
 2017-06-19 20:38:12 -05:00
tritoch
4f49559751 Missed file extension 2017-06-19 15:57:11 -05:00
tritoch
ba87735c6a Merge branch 'master' into split-cards-and-xml-dump 2017-06-19 15:31:35 -05:00
tritoch
fbdee30d65 Improved Split Card Handling. Dump XML to log. 2017-06-19 15:20:48 -05:00
tritoch
ef4f2c7377 Improved Split Card Handling. Dump XML to log. 2017-06-19 14:49:00 -05:00
tritoch
4ee8c132eb Improved Split Card Handling. Dump XML to log. 2017-06-19 14:45:32 -05:00
tritoch
094ca85029 Allow comments in JSON files 2017-06-19 12:12:55 -05:00
tritoch
aac7664caf Revert "Allow comments in JSON files" 
This reverts commit 9f1b096e0c.
 2017-06-19 13:06:40 -04:00
tritoch
9f1b096e0c Allow comments in JSON files 2017-06-19 11:55:09 -05:00
tritoch
aa7ea2270b Return of Split Cards. MPS set starting. Sanitize Scryfall. 2017-06-17 00:10:52 -05:00
tritoch
f7d1f54784 Multiple Set Support 2017-06-16 09:55:45 -05:00
tritoch
997ef150a6 Not Full Spoil 2017-06-13 15:43:30 -05:00
tritoch
f038522634 Fix XML Pretty Print 2017-06-13 15:24:59 -05:00
tritoch
19add21493 Enable Scryfall - Just use for Diff for now. 2017-06-06 15:30:38 -05:00
tritoch
ccd094f37c Turn MTGS back on. Small format fix for XML. 2017-06-06 15:03:37 -05:00
tritoch
a01b08558e Pretty spoiler.xml 2017-06-01 20:12:44 -05:00