Nekkowe!

The FF14 Crystarium Inn Room in SteamVR Home

Written 2024-10-31, backdated to 2021-03-25 (date of project completion).

I love taking Things From Games and putting them Elsewhere....... I love it sooo much.... and it only gets easier as time passes, with standardized formats and useful community modding tools for 3D models & textures. Unfortunately, FF14 ARR released in like 2013, and not all quirks are really accounted for...

In the Shadowbringers expansion of Final Fantasy XIV, you get assigned a really nice inn room in the residential "Pendants" area:

It's a very pleasant place, with a bunch of empty space. I'd been dipping my feet into VR and vaguely playing with the idea of modeling a custom SteamVR Home room for myself in Hammer with the provided Source 2 tools, but it seemed even more appealing to try and rip this existing room I already liked out of its game and wrangle it into SteamVR Home as a place to inhabit.

At the time, I'd been trying out various tools for looking at the ingame enemy models and level geometry. The main three useful tools for inspecting and exporting those are/were:

For this project, I used Godbert to extract the level geometry.

As a general note on extracting FF14 game data to use in silly projects - this was all before the Dawntrail graphics update. At the time of writing, my sub is paused, so I'm not sure if the texture format has changed. But at the time, it was kind of horrible to work with. It's all these .DDS files with some really weird channel mapping for everything but the diffuse textures.

The .DDS texture format

The .DDS format itself is the easy part. For viewing/editing .DDS files, I recommend GIMP or ImageMagick. I think Blender can handle importing them, too? Krita unfortunately doesn't open these.

At the time of the project, I was still using Windows. You can view .DDS thumbnails in Windows Explorer if you install NVIDIA's "DDS Thumbnail Viewer" thing. I hear SageThumbs also works, but I haven't tried that one out myself.

At the time of writing this post, I've mostly ditched Windows in favour of Linux Mint. XnView MP is free for personal use and has turned out to be extremely helpful for quick thumbnail overviews of a folder full of these things. It has a lot of features I haven't taken the time to get to grips with, yet, but if I end up returning to a project adjacent to all this stuff again, I'll likely be making extensive use of it.

But with the easy part out of the way,

What in the world are these channels

I have next to no knowledge of how exactly the shader works internally - it's a specular workflow, but also not quite? There's this weird mask texture... most-but-not-all of the normal maps are like this:

A folder full of .DDS files extracted from FF14. All the normal maps are oddly green.

As far as people have been able to figure out, the blue channel in FF14's normal maps is responsible for opacity[1]. Which at the time, I tried to "correct" for with a python script like so:

[...]

def normalize_two_channel_normal_map(two_channel_normal_map: Image) -> Image:
    bands = (red, green, blue, alpha) = two_channel_normal_map.split()

    blue_new = Image.fromarray(
        np.full((
            blue.size[1],
            blue.size[0]
        ), 255)
    ).convert("L")

    return Image.merge("RGBA", (red, green, blue_new, alpha))

[...]

(Mind you, at the time, I didn't know a thing about colour spaces or how unaware Pillow is of them by default. I still barely do. There's a chance that somewhere along the conversion from .DDS to .PNG, I mishandled things in a way that at least in part affected the final results? How do graphics programmers deal with any of this on the regular without going insane..?)

Not my brightest moment, I think. I mean, clearly, the blue channel going in isn't just zero across the board. There's gotta be some information I'm losing there. But I can't make any more sense of what's going on with these now than I could back then. Heck, I think this was me copying the way that TexTools did it at the time? Or still does...? It's been way too long. I'm digging through my files and can't figure out my reasoning here anymore. What's up with these files?

But then you have the really bad part: these whacky specular maps with ??? what ????

Most of the modding tools only worry about characters, gear and furniture, so I have absolutely no idea whether the environment shader handles the mask texture the same way as e.g. the furniture does. If it does do so, then the red channel is a "Diffuse Mask"(?)[2], green is a specular mask (but completely absent in just about every single such texture for the inn room? Even though there's lots of stuff made of metal etc. in there...?) and blue is gloss, with the caveat of:

SE's Gloss implementation is very unique, and doesn't adhere particularly well to most modern PBR system's assumption of Gloss. (Closer to non-PBR Spec-Gloss)

Very unique, great! That's great, except for the part where I'll eventually have to get these to work with existing Source 2 shaders that won't be accomodating any unique quirks of FF14's channel-packing or rendering methods.

At the end of the day, I set the blue channel to a full 255 on every normal map - as seen in the script above - and otherwise left the textures untouched. The result is decidedly iffy. Any nice shiny and/or metallic surfaces just look like flat rough papercraft at best. The normals seem generally kind of off.

But the more I type this all out, the more tired I get of worrying about these formats. I caught myself wanting to go back and do it all over again (followed by revisiting three other projects involving FFXIV model extraction), can you imagine? Oh my god. I have to get to the interesting part already and rush to wrapping this up. I'm losing my gourd. It's 4am and I've forgotten every meal since yesterday's breakfast because I was preoccupied with work and with styling this blog.

Anyway, specular aside, I got all that stuff into Blender more or less fine:

Now with the models and textures in hand

The next rough part was getting the models into SteamVR.

Different Source 2 games come with different sets of tools. The Source 2 tools for Half Life Alyx are incredible and have wonderful support for all sorts of model and texture formats right out the gate. On the other hand, I have repressed most memory of what I had to do to get these resources into the SteamVR Home. The model import process and everything to do with shading is worse to an unbelievably degree.

Similarily, once you do import everything, SteamVR Home has a very slapdash approach to various things...

...that HL:A handles perfectly well right out the gate:

In that particular case, it was light leaking through backfaces, culling or no.
I ended up taking the route of just applying a Solidify modifier in Blender. It took a bunch of fixes like that. But then:

The immediate next problem was that the whole thing froze up the moment any physics came into play - the models were relatively detailed and only ever intended for display, never to serve as a collision box.

Which meant modeling rudimentary collision hulls for every single piece of moderately convoluted furniture, going from this:

...to this:

...which fixed all performance issues immediately.

The Lighting

You'll notice the lighting is very different from the actual ingame inn room, and that's partly to do with FF14's mysterious shaders and partly with its mysterious lighting information.

Godbert does also export lighting info, which for the Pendants inn room looks something like this:

Click to open:
#LIGHT_7__7841572
#pos -5,799361 1,00917 -0,3313377
#UNKNOWNFLAGS 0x00000000 0x0000FF7E 0x00000101 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 0,5 45
#unk3 1,2 0
#unk4 0,01 0

#LIGHT_8__7872634
#pos 0 4,450831 -7,672129
#UNKNOWNFLAGS 0x00000000 0x0000FFF7 0x00000101 0x00000000
#UNKNOWN 0,978443 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 60 60
#unk3 4 0
#unk4 0,01 0

#LIGHT_9__7924149
#pos 4 7,3 0
#UNKNOWNFLAGS 0x00000000 0x0000FFAC 0x00000101 0x00000001
#UNKNOWN 1,570796 0 -0,1396263
#UNKNOWN2 1 1 1
#unk 2 1
#unk2 20 80
#unk3 8 0
#unk4 0,55 0

#LIGHT_10__7924170
#pos -4 7,3 0
#UNKNOWNFLAGS 0x00000000 0x0000FFAC 0x00000101 0x00000001
#UNKNOWN 1,570796 0 0,1396263
#UNKNOWN2 1 1 1
#unk 2 1
#unk2 20 80
#unk3 8 0
#unk4 0,55 0

#LIGHT_11__7924187
#pos -8 3,17136 -5,906682
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000101 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 0,5 45
#unk3 2 0
#unk4 0,3 0

#LIGHT_12__7924205
#pos -8 3,17136 6
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000101 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 0,5 45
#unk3 2 0
#unk4 0,3 0

#LIGHT_13__7924399
#pos 8 3,65749 6
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000101 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 0,5 45
#unk3 2 0
#unk4 0,3 0

#LIGHT_14__7924416
#pos -0,03024244 8,216222 -1,468262
#UNKNOWNFLAGS 0x00000000 0x0000FFFF 0x00000101 0x00000000
#UNKNOWN 2,792527 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 60 60
#unk3 3 0
#unk4 0,01 0

#LIGHT_15__7925125
#pos 11,49656 1,548322 -1,890859
#UNKNOWNFLAGS 0x00000000 0x0000FF7E 0x00000101 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 0,5 45
#unk3 1,2 0
#unk4 0,01 0

#LIGHT_16__7925128
#pos 6,155866 1,952266 5,449976
#UNKNOWNFLAGS 0x00000000 0x0000FF7E 0x00000101 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 0,5 45
#unk3 1,2 0
#unk4 0,3 0

#LIGHT_17__7926433
#pos 8 3,576245 4,791916
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000101 0x00000000
#UNKNOWN 0,7452181 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 40 40
#unk3 3 0
#unk4 0,01 0

#LIGHT_18__7926440
#pos -8 3,576245 4,791916
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000101 0x00000000
#UNKNOWN 0,7452181 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 40 40
#unk3 3 0
#unk4 0,01 0

#LIGHT_19__7926482
#pos -7,692634 1,00917 4,873642
#UNKNOWNFLAGS 0x00000000 0x0000FF7E 0x00000101 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 0,5 45
#unk3 1,2 0
#unk4 0,01 0

#LIGHT_20__7927505
#pos 5,5 5 0
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000001 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 2 1
#unk2 0,5 45
#unk3 3 0
#unk4 0,01 0

#LIGHT_21__7927572
#pos -5,5 5 0
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000001 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 2 1
#unk2 0,5 45
#unk3 3 0
#unk4 0,01 0

#LIGHT_22__7931550
#pos -8,727137 3,072884 2,071778
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000101 0x00000000
#UNKNOWN 0,1006231 -1,274564 0,4306677
#UNKNOWN2 1 1 1
#unk 2 1
#unk2 60 60
#unk3 1,2 0
#unk4 0,01 0

#LIGHT_23__8044407
#pos -10,7128 0,7905494 -3,393785
#UNKNOWNFLAGS 0x00000000 0x0000FF1E 0x00000101 0x00000001
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 3 1,5
#unk2 0,5 45
#unk3 1,1 0
#unk4 0,01 0

#LIGHT_24__8044524
#pos 8,939381 5,36302 2,718756
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000001 0x00000000
#UNKNOWN 1,198634 0,2078233 0,2394475
#UNKNOWN2 1 1 1
#unk 3 1
#unk2 20 40
#unk3 4 0
#unk4 0,01 0

#LIGHT_25__8044821
#pos 5,5 8,301979 0
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000001 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 1 1
#unk2 0,5 45
#unk3 0,8 0
#unk4 0,01 0

#LIGHT_26__8044822
#pos -5,5 8,302 0
#UNKNOWNFLAGS 0x00000000 0x0000FFBB 0x00000001 0x00000000
#UNKNOWN 0 0 0
#UNKNOWN2 1 1 1
#unk 1 1
#unk2 0,5 45
#unk3 0,8 0
#unk4 0,01 0
n4ti-n4ti_room_light-lights.txt

Now, that format struck me as somewhat less than helpful. The positions make sense, at least:

But even just figuring out which of these lights is meant to be a point light, some other sort of light, what colour it ought to be, how bright... I did actually try reverse-engineering what the flags meant for a while, in the end: But it wasn't very fruitful. Even by the end, I didn't know my UNKNOWNFLAGS from my UNKNOWNs, my UNKNOWN2s, my unks nor my unk2s. So I placed some lights by hand in SteamVR Home's Source 2's Hammer editor and called it a day.

If you know more about this lighting format, please do get in touch. I don't think I'll ever want to revisit this particular project, but there are a couple other ones I've shelved in the past that would benefit from being able to recreate the lighting more closely in Blender.

In Conclusion

When all is said and done, once the aforementioned texture nightmare and the initial Source 2 model import process were sorted, I had a really nice time with the back half of the project.

It's completely possible to extract level geometry & texture data from FF14 and walk around inside it in SteamVR home. I've been asked if it's also possible to bring places into VRChat by the same method, and I'm sure it is possible for the most part, but I can't really speak on how much effort it'd be, since I have absolutely no idea what restrictions on shader stuff VRChat has...

My biggest takeaway from all this was a newfound appreciation for the metallic/rough PBR workflow, as opposed to dealing with this extra-weird flavour of specular / gloss stuff. I was incredibly out of my element start to finish.

One day, I want to dig a lot deeper into shader programming and the history behind all this stuff - I'm hoping it'll give me a stronger understanding of what exactly I was dealing with here, and why e.g. those weird masks were the way they were.

Post-Endwalker burnt me out on FF14 pretty badly, and Dawntrail didn't really help that, so I've paused my sub and don't have a lot of motivation to delve back into FF14's models and textures and look into what people have found out about how its shaders work post-graphical-update. It seems like in the meantime, a lot of helpful resources have been provided by the FF14 modding community (though seem to put a lot more focus on the characters than the surrounding world). Maybe one day.

  1. Despite the diffuse maps already having a perfectly fine & actively utilized alpha channel. ↩︎

  2. I'm still genuinely not sure what this is for. Maybe some sort of baked ambient occlusion...? ↩︎