Ys VIII Data Extraction

This is a place to post tech details and informative documents about romhacking PSP games.
User avatar
NightWolve
Posts: 63
Joined: Fri Apr 17, 2015 10:32 am
Location: Chicago, IL, USA
Contact:

Re: Ys VIII Data Extraction

Post by NightWolve » Mon Aug 08, 2016 10:58 pm

Hm, that's the same storage strategy that was only used with Ys II Complete by the sound of it. The bitplane data of the BMP was stored separate from the palette header data (which all went into one file) - guess they felt they gained extra compression if so many BMP files used the same palette. They only used that once though, then went on to the 3D Ys VI engine where using NVIDIA's DDS format was the standard combined with using standard ZLIB to zip it up.

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: Ys VIII Data Extraction

Post by flamethrower » Tue Aug 09, 2016 12:19 am

I did it!

It's Aaron, leader of Drifter's Village.
dr_aaron.png
dr_aaron.png (22.41 KiB) Viewed 682 times
Now that we're through celebrating, it's time to learn something. Never a dull moment :)

This guy wrote or copied (or whatever) unswizzle code:
http://forum.xentax.com/viewtopic.php?p=21148#p21148

I translated the code into python, ran it, and it worked.
He posted that code in 2007, I'm surprised it still works.
This is mine:

Code: Select all

with open('bitmap.bin', 'rb') as f:
    filedata = f.read()
buffer = bytearray(len(filedata))
readoffset = 0

def unswizzle(writeoffset, segwidth, segheight, datawidth):
    global buffer, readoffset
##    print(segwidth, segheight)
    if segwidth == 2 and segheight == 2:
        buffer[writeoffset:writeoffset+2] = filedata[readoffset:readoffset+2]
        buffer[writeoffset + datawidth:writeoffset + datawidth + 2] = filedata[readoffset + 2:readoffset + 4]
        readoffset += 4
    else:
        unswizzle(writeoffset, segwidth // 2, segheight // 2, datawidth)
        unswizzle(writeoffset + segwidth // 2, segwidth // 2, segheight // 2, datawidth)
        unswizzle(writeoffset + datawidth * (segheight // 2), segwidth // 2, segheight // 2, datawidth)
        unswizzle(writeoffset + datawidth * (segheight // 2) + segwidth // 2, segwidth // 2, segheight // 2, datawidth)

unswizzle(0, 256, 256, 256)
with open('bitmap3.bin', 'wb') as f:
    f.write(buffer)
Is it weird if it's rotated like that? Did I do something wrong?

My god that's hard to wrap your brain around.
Your image is a power of two square in both dimensions. Basically you keep subdividing the [square] image into 4 smaller squares. When you get to a square that's 2x2, the pixels are stored in top left, top right, bottom left, bottom right order. If you know that, you put them in the right places in a linear array so it can be put into normal image formats.

Now I gotta learn how to use PyPNG.
I gotta research how to marry up a color table and bitmap into some kind of image format. That image was generated using TiledGGD.

User avatar
NightWolve
Posts: 63
Joined: Fri Apr 17, 2015 10:32 am
Location: Chicago, IL, USA
Contact:

Re: Ys VIII Data Extraction

Post by NightWolve » Tue Aug 09, 2016 1:07 am

Congrats! Or "EUREKA! EUREKA!" :)

josejl
Programmer
Posts: 177
Joined: Mon Apr 13, 2015 6:49 pm
Location: Spain
Contact:

Re: Ys VIII Data Extraction

Post by josejl » Tue Aug 09, 2016 12:03 pm

I think I got the rotated texture when I did the PhyreEngine png converter, because there were two kind of settings for the unswizzle function, YX and XY.

User avatar
BatteryShark
Programmer
Posts: 69
Joined: Wed Dec 16, 2015 12:06 am

Re: Ys VIII Data Extraction

Post by BatteryShark » Fri Sep 02, 2016 3:21 pm

I started updating my script disassembler/assembler:

- A few new opcodes, no surprise there
- The talkmes function has been replaced with a new one "UNK_8126"... guess it's not really UNK... lol. They actually structure their lines now which is kinda nice.
- Couple new data types (string array is one) with a few new flags, besides that, the few I've tried have disassembled pretty nicely, should assemble fine as well.

Check out the attached disassembly of mp1116.

Cheers,
Attachments
mp1116.zip
(7.99 KiB) Downloaded 18 times

User avatar
BatteryShark
Programmer
Posts: 69
Joined: Wed Dec 16, 2015 12:06 am

Re: Ys VIII Data Extraction

Post by BatteryShark » Tue Sep 06, 2016 5:46 am

Disassembler is fully working now with *almost* everything named, one datatype I think is 'tan', but not sure - use this opcodes.py with my old celceta disassembler:

https://gist.github.com/anonymous/b9062 ... d4e923ca57

I still have to add the stringarray datatype for reassembly... that should be quick.

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: Ys VIII Data Extraction

Post by flamethrower » Tue Sep 06, 2016 1:54 pm

One thing I noticed is they do not use data compression.
Would loading times be shorter if they used data compression?
The characteristics of the medium (Vita Game Card or Vita Memory Card) might be significantly different from PSP UMD or PS3 Game Disc, meaning data compression isn't as useful.

josejl
Programmer
Posts: 177
Joined: Mon Apr 13, 2015 6:49 pm
Location: Spain
Contact:

Re: Ys VIII Data Extraction

Post by josejl » Tue Sep 06, 2016 2:09 pm

That's an interesting question, I guess it depends on what's causing the bottleneck, card reading speed or CPU.
If we ever getting proper overclocking on Vita we'll know for sure.

User avatar
BatteryShark
Programmer
Posts: 69
Joined: Wed Dec 16, 2015 12:06 am

Re: Ys VIII Data Extraction

Post by BatteryShark » Tue Sep 06, 2016 8:05 pm

In terms of graphics, the additional I/O reads are normally negligible compared to the overhead of pulling it compressed into memory, extracting it, and then performing conversions... normally faster to pump an uncompressed texture to the GPU. Now if you're loading a shit ton of textures at once, you might run into memory issues with that while you're waiting for stuff to be processed. I'd say it really depends, but at the end of the day, you'd probably want to keep textures close to what is finally loaded as possible. I've seen arcade games with tons of textures keep everything uncompressed (e.g. Wangan Midnight MaxTune4 is like 60GB because of all the damn textures, yet compressed it's like close to 7 or so).

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: Ys VIII Data Extraction

Post by flamethrower » Sun Oct 23, 2016 1:23 am

Here is my Ys8 it3cnv. It also does itpcnv.
it3cnv_Ys8.7z
(2.68 KiB) Downloaded 26 times

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: Ys VIII Data Extraction

Post by flamethrower » Tue Oct 25, 2016 4:01 am

Trying to make nicely formatted dump with only character and user text data.

Opcodes likely to be important:
0x8126 TalkPopup <- The big one!
0x8040 TalkMes
...I guess that's it. So now to inspect those opcodes...

8040:
0: ChrName
1: String0
2: String1
3: String2

8126 (harder...but not by much I guess):
0: ChrName
1...6: Unknown
6: StringArray
----------------
Alright, making good progress on this so far.
Next will be to clean / separate the # codes and then it's good, I think.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest