PSN Extract

PSN Extract


PSN Extract is a tool intended to gather game trophy data from PlayStation Network, parsing downloaded html for use building external web pages.

PSN Extract at GitHub




NAME

psnextract


SYNOPSIS

Firefox save trophy game page from PSN US:

 save-as web-page-complete: gamedata-us.html (gamedata-us_files)

Parse data and output to webdir with included graphics:

 psnextract.pl --us gamedata-us.html --include includedir  --web outputdir

Combination with UK data:

 psnextract.pl --us gamedata-us.html --uk gamedata-uk.html  --include includedir --web outputdir

Designate file for override variables:

 psnextract.pl --us gamedata-us.html --uk gamedata-uk.html  --include includdir --override override.txt --web outputdir

DESCRIPTION

Psnextract is a tool intended to gather trophy data from PlayStation Network, parsing downloaded html for use building external web pages. It is able to parse and merge output from both the US and UK sites, and data can be overridden at will.


INSTALLATION

Script requirements are as follows:

 HTML::TokeParser::Simple;

Installation consists of merely untarring the script, and likely making use of the included graphics with the include argument. At this time, html output formatting and styles are hardcoded, so any customization must be done via programmatic changes within the script.


OPTIONS

-d, --dry-run

Do not actually do any operations. Combines well with --verbose to debug problems.

-h, --help

Print brief usage message.

-i, --include=/path/to/include/graphics

Path to directory of graphics and other items to include when building web. The include directory provided in the package includes all items necessary for building web pages as is done in the write_html function. Adjustments to that function may well require additions to include.

-o, --override=/path/to/override

File of corrections and additions with which to override game and trophy values parsed from html. Format is item|attribute=value. See OVERRIDES.

--uk=/path/to/uk_psn.html

File of UK PSN html save-as web-page-complete data to parse. Assumes Firefox format, with file uk_psn.html and accompanying directory uk_psn_files.

--us=/path/to/us_psn.html

File of US PSN html save-as web-page-complete data to parse. Assumes Firefox format, with file us_psn.html and accompanying directory us_psn_files.

-v, --verbose

Control vebosity of output. Increase by passing multiple times.

-w, --web=/path/to/webdir

Web destination to which include graphics and items are copied, and inside which index.html is written from parsed data.


HTML SOURCES

Source HTML data is that seen when browsing to specific game/trophy progress when logged into us.playstation.com and/or uk.playstation.com, and saving the web page with Firefox 'web-page-complete' facility. This results in an .html file and accompanying _files directory containing graphics and css.

The reason for supporting both US and UK trophy data sources and features for merging them and overlaying overrides is the following observation:

 US: Datestamps, bigger trophy graphics, often missing or delayed DLC
 UK: No datestamps, abbreviated trophy titles, always up-to-date DLC 

The main issue is missing DLC. As such, the sources are overlayed on top of one another: UK -> US -> overrides. UK data to provide data for all trophies minus datestamps, US to fill in missing datestamps and provide nicer graphics, and lastly overrides to fill in any missing info.

In most cases, US data is all that is needed to provide all info.

NOTE: In the current version of the US site, ensure you scroll down to click 'MORE' to show all trophies before saving the page. Otherwise the div does not contain all information and trophies will be missing.


OVERRIDES

The overrides facility is used to provide missing info, corrections, addendums, and any caption desired for web output. Overrides are one per line, in format item|attribute=value. Example specifying most possible overrides:

 game|title=>Game of the Ages
 game|caption=>This is a terrific game
 game|img=>awesome.jpg
 game|user=>foobar
 game|avatar=>foobar.jpg
 game|progress=>50
 game|bronze=>12
 game|silver=>34
 game|gold=>56
 game|platinum=>78
 1|date=>Fri Aug 24 22:49:00 EDT 2012
 2|date=>Fri Aug 24 20:45:00 EDT 2012
 3|date=>Fri Nov 12 22:22:00 EST 2010
 4|date=>Fri Aug 31 21:44:00 EDT 2012
 5|date=>Sun Sep 16 15:15:00 EDT 2012

Every game line is overriding info that will appear in the masthead, while numbered lines provide info for the given trophy. It is most common to use the overrides file only to specify caption, and in the event of missing DLC, trophy dates.


EXAMPLES

Parse US PSN data:

 psnextract.pl --us=gamedata_us.html

See lots of verbose output:

 psnextract.pl -v -v -v -v --us=gamedata_us.html

Also parse UK PSN data:

 psnextract.pl --us=gamedata_us.html --uk=gamedata_uk.html

Add in overrides:

 psnextract.pl --us=gamedata_us.html --uk=gamedata_uk.html  --override=override.txt

Now build a web page with it all:

 psnextract.pl --us=gamedata_us.html --uk=gamedata_uk.html  --override=override.txt --web=webdir

Include graphics for complete html output:

 psnextract.pl --us=gamedata_us.html --uk=gamedata_uk.html  --override=override.txt --include=includedir --web=webdir

CHANGES

21041007

-Complete perldocs, pod2text README

93c715124cb18143d9b02ffc5363b75f366a7c89 (20140927)

-Initial release