IT3/ITV Formats

A forum for the translation project of Nayuta no Kiseki.
User avatar
SkyeWelse
Graphics & Web Design
Posts: 478
Joined: Thu Mar 05, 2015 4:32 am
Location: Georgia
Contact:

Re: IT3/ITV Formats

Post by SkyeWelse » Sun Jun 07, 2015 6:30 pm

Yeah I found earlier that if you remove the reference of mmv3/b005.itv that is being called from scripts/mp_8098.bin, it will still display the ITV file packed in the pack/scripts/mp_8098.2pp file in the center of the screen and make it look distorted. So I saw that it was definitely controlling at least the position and frame size of that itv file.

I'll make a note of that translation of the area name by the way. Thanks for letting me know. It would be difficult to correct these later after I start them. They would pretty much have to be started over from scratch most likely. The reason for this is mainly because it would be too chaos-like to have so many layers and groups of pieces of the graphic sitting above each frame. I'll explain to see if it makes more sense.

So I have a group of layers as frames for the entire animation. I pick out elements that will be destroyed by removing the Japanese text ontop of them, gears, black bars, parts of the animation that would need to be recreated and build them all over again. I keep the new assets inside a folder group with each part labeled. Then I also pick the Japanese characters pixel-by-pixel and save these graphics out in this group so that I can use them purely for "selection" purposes. So... In each frame, I select the Japanese using the graphics that were cut out for selection purposes, and I turn on the layers of the rebuilt graphics that were destroyed and adjust the opacity to match the original Japanese graphic. Each time I do this and get it just right, I make a copy of the entire group and move it to be above the next frame I wish to work on. Then with the existing frame I merge them all into one layer and make sure it's named appropriately.

Then starting on the next graphic / layer in sequence I repeat the process again, selecting the Japanese to remove, and fixing the broken pieces and adjusting opacity, repeat. If I didn't merge the layers, I'd have 89 groups of files/pieces, etc sitting above each of the 89 frames/layers which would be a bitch to work with. : )

The Nayuta Tutorials were like this though, and I had so many layers and stuff ontop of each other that my computer hated me for it. It made Photoshop run pretty slow at times, and I have a fairly nice machine bought in the last year and half with 16 GB Ram and an SSD...

-Thomas

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: IT3/ITV Formats

Post by flamethrower » Sun Jun 07, 2015 7:04 pm

About the area name, it displays when you push select when fighting the boss. This data is textual; I don't think it affects what you're doing.
NPJH50625_00091.jpg
NPJH50625_00091.jpg (51.71 KiB) Viewed 977 times
This should read "Astrolabe/Interior"

Did OMGfloofy say they want to pick up the project when we're through? Because we're pretty much through at this point. You should contact her to see what she thinks it should say.

User avatar
SkyeWelse
Graphics & Web Design
Posts: 478
Joined: Thu Mar 05, 2015 4:32 am
Location: Georgia
Contact:

Re: IT3/ITV Formats

Post by SkyeWelse » Sun Jun 07, 2015 11:24 pm

I see what you mean. Astrolabe / Interior.

I had asked Floofy before if she wanted to be a translation checker for Nayuta, but I don't think she was interested anymore. If it were to get a translator on board, it would likely be someone else.

But yeah, other than these graphics needing to be completed and tested in the game, I'd say the project is done.

Edit: First 50 Frames of Animation for Labe Virgos.

Image

-Thomas

User avatar
SkyeWelse
Graphics & Web Design
Posts: 478
Joined: Thu Mar 05, 2015 4:32 am
Location: Georgia
Contact:

Re: IT3/ITV Formats

Post by SkyeWelse » Sat Jun 13, 2015 6:58 pm

First one down! Only... 1, 2... 13.. 32 more to go!



-Thomas

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: IT3/ITV Formats

Post by flamethrower » Sat Jun 13, 2015 7:05 pm

I use Open Broadcasting System which is free open-source software. Yours works. The advantage of OBS is that it avoids the watermark. I'm not expert in video recording software. Bandicam might have advantages too.

For demos, you should switch to 4.14 because it has the text speed fixed.

You shouldn't feel rushed. Maybe take a break by working on Wander Wonder or something.
Now you know how the Zero translators feel. :)

User avatar
SkyeWelse
Graphics & Web Design
Posts: 478
Joined: Thu Mar 05, 2015 4:32 am
Location: Georgia
Contact:

Re: IT3/ITV Formats

Post by SkyeWelse » Sat Jun 13, 2015 7:24 pm

Cool, I'll give Open Broadcasting System a try. I was thinking of purchasing Bandicam since I do like how easy to use it is for recording and taking screenshots (say with Wander Wonder without needing to run it in DxWnd) but I also like "Free" too. : )

Yeah, I'll kind of work on these little-by-little I think, but it's nice to know that they can be edited bearing that someone has enough time to edit them all... lol

-Thomas

josejl
Programmer
Posts: 177
Joined: Mon Apr 13, 2015 6:49 pm
Location: Spain
Contact:

Re: IT3/ITV Formats

Post by josejl » Thu Aug 11, 2016 10:25 am

I've been trying to research the IT3 format lately.

Basically,I've run into a lot of issues regarding Celceta's shadows, they don't work correctly in some levels, and it's been troubling me for months now.
post-239802-0-45538200-1470569722.jpg
post-239802-0-45538200-1470569722.jpg (129.65 KiB) Viewed 690 times
As you can see the shadows get casted on the floor ,but they also get casted below,as I crudily marked. The vertex normals data is totally off when I checked it,so no wonder it doesn't work properly, but , how did Falcom get it to work there?

I should have asked that question to myself way earlier, instead of 3 days ago. They simply disabled the shadows in the Vita version in that level.

So, basically,if I want the shadows to work there as well,I'd need to fix the vertex data, which is located in the VPA8 sections of the IT3 files. (With this out of the way, I think that making an IT3 importer/exporter would be feasible, the other sections seem easier to deal with)

However, I am totally lost at this one, data is packed in a really strange way which doesn't make any sense to me, and reversing the code for loading these doesn't help much either.

Did anyone ever get a response from pokanchan? I think I wrote him once a few years ago about the Brandish 3 compression , but he never replied. He's the only one who's made a converter for these files that I know.

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: IT3/ITV Formats

Post by flamethrower » Thu Aug 11, 2016 1:04 pm

Maybe I could try leaving a comment on his blog?
http://www.pokanchan.jp/dokuwiki/blog/start
I'll just leave it in the latest entry. We want to ask for source code for IT3CNV, correct?

My Japanese is bad but it should be understandable.

SkyeWelse left a comment here and I'm sure he saw it. https://pledgie.com/campaigns/19818
One of those donations is mine, the itpcnv tool proved invaluable for the Nayuta project.

The formats look the same as earlier games but I don't really know. So one thing to try would be to look at the VPA8 loader code in a Falcom PSP game to see if it makes any more sense. I'll see if I can find a breakpoint address for you.

I'm afraid I don't know much about 3D graphics.

I do have a hint for you though. Some of the data in the VPA8 blocks is compressed with FALCOM3. You didn't mention it in your post so maybe you weren't aware of it. Maybe it would be instructive if you unpacked it.
For example, in Celceta\map\mp1101\mp1101.it3, there is a VPA8 block at 0x368. The FALCOM3 stream starts at 0x37C. That first integer 0x250 is the stream length so you can see how long it is. There is another compressed stream in the same block at 0x5D0.

FALCOM3 goes:
FALCOM3 compressed length (including this header), uncompressed length, number of FALCOM2 streams
then, repeating for each FALCOM2 stream:
Halfword: FALCOM2 stream length
FALCOM2 stream
Byte: 0x01 for "there are additional chunks past this one" or 0x00 for "this is the end of the stream"

You can use my FALCOM decompression library for Python: http://pastebin.com/WpWJHu6B
Mine is based on EsperKnight's code, see below.
You can use EsperKnight's decompression routine source code (written for some flavor of C), which is built for FALCOM2. FALCOM3 is a wrapper that chains multiple FALCOM2 streams together. You can download EsperKnight's code here: http://www.romhacking.net/forum/index.p ... #msg256702

josejl
Programmer
Posts: 177
Joined: Mon Apr 13, 2015 6:49 pm
Location: Spain
Contact:

Re: IT3/ITV Formats

Post by josejl » Thu Aug 11, 2016 2:31 pm

flamethrower wrote:
I do have a hint for you though. Some of the data in the VPA8 blocks is compressed with FALCOM3. You didn't mention it in your post so maybe you weren't aware of it. Maybe it would be instructive if you unpacked it.
For example, in Celceta\map\mp1101\mp1101.it3, there is a VPA8 block at 0x368. The FALCOM3 stream starts at 0x37C. That first integer 0x250 is the stream length so you can see how long it is. There is another compressed stream in the same block at 0x5D0.

FALCOM3 goes:
FALCOM3 compressed length (including this header), uncompressed length, number of FALCOM2 streams
then, repeating for each FALCOM2 stream:
Halfword: FALCOM2 stream length
FALCOM2 stream
Byte: 0x01 for "there are additional chunks past this one" or 0x00 for "this is the end of the stream"

You can use my FALCOM decompression library for Python: http://pastebin.com/WpWJHu6B
Mine is based on EsperKnight's code, see below.
You can use EsperKnight's decompression routine source code (written for some flavor of C), which is built for FALCOM2. FALCOM3 is a wrapper that chains multiple FALCOM2 streams together. You can download EsperKnight's code here: http://www.romhacking.net/forum/index.p ... #msg256702
Oh,thanks, that was quite helpful, I had no idea about that. It totally makes sense with what I have seen in the code,the first 12 bytes are read first, and then the FALCOM3 part. No wonder it didn't make much sense to me, I wasn't expecting normal compression for this, I guess it caught me offguard.

I still can't understand the uncompressed data, but at least I can see most of the data I know it should be there.

flamethrower
Programmer
Posts: 783
Joined: Mon Mar 09, 2015 3:03 pm

Re: IT3/ITV Formats

Post by flamethrower » Sat Aug 13, 2016 9:15 pm

I made a data visualization tool for you.
And also I figured out some of the values.

IDSTRING VPA8
Word data_len
Word unknown_0x8 section 0 length divided by 116 (also equals section 2 length divided by 12)
Word unknown_0xC section 1 length
Word unknown_0x10 2 times section 3 length
FALCOM3 stream -> Section 0 (116-byte entries)
FALCOM3 stream -> Section 1 (40-byte entries) (might be more than one stream, the limit for a single FALCOM3 stream is 0x40000)
FALCOM3 stream -> Section 2 (12-byte entries)
FALCOM3 stream -> Section 3 (halfword entries)

We just need to understand the content of these sub-files.

Okay, here's the program:
VPA8_vis.7z
(1.62 KiB) Downloaded 28 times
Sample output: http://pastebin.com/zCKf90rq
----------------------------------------------------------
I read about .obj and .mtl files on Wikipedia. You know, the ones that the it3cnv tool outputs.
Wikipedia link: https://en.wikipedia.org/wiki/Wavefront_.obj_file

These files are defined by:
Lists of vertices
Lists of vertex indices

That sounds like what's actually going on here. Section 1 has the vertex list and Section 3 has the vertex indices (that's just a guess, I don't really know). I can't match up the values in section 3 with the values in section 1. The highest referenced value in s3 is well short of the number of vertices in s1.

josejl
Programmer
Posts: 177
Joined: Mon Apr 13, 2015 6:49 pm
Location: Spain
Contact:

Re: IT3/ITV Formats

Post by josejl » Sat Aug 13, 2016 10:14 pm

Seems you beat me to it.
I was starting to look Section 2 when you posted your results.
I don't know what Section 0 is,really. The game just loads the data, and I haven't looked what it does with it later.

Section 1 has the vertex info

Float3 Position
Float2 Texcoords (the Y coordinate is negated in IT3CNV, depends on the rendering API actually)
Int4 RGBA color, probably diffuse color. Most of the time it's (127,127,127,255) Probably has something to do with the PSP games working at 16 bit colour depth.
8 bytes, that are always at zero so far. Looking at the calculations it does during loading, it looks like colour info too.
4 bytes, always 0x00000080 at the files I've seen. It's got something to do with the colours before. Possible vertex blending data (when you blend between 2 different textures, for example, grass and ground textures)
4 bytes I don't have a clue of what really they do,except that it seems to be colour related as well.

No trace of the vertex normals, which are the thing I was looking for. Those can be calculated in loading time, but seeing as some the data is wrong, I wouldn't say so.


Section 3 has got the indices, 2 bytes each.

These are used to speed up rendering (depends on the hardware,actually) and save RAM. 3D rendering most of the time is done by triangles. Say you want to render a square, you need 2 triangles for that.

Each triangle needs to be defined by 3 points, therefore, you need to use 6 vertices in order to render them, even if two of these are the same.

With an index,you can write only the necessary vertices, and then define the triangles with those. Something like draw a triangle with the vertices 1,2 and 3 and another one with vertices 1,2 and 4.

No idea about Section 2 so far.

Edit: Section 2 seems to have a bunch of zeroes,and the count of indices in Section 3 (Basically,Section 3 length divided by 2)

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest