Use a more correct atlas list

[Nightwarden24]

Description

Your way of generating a texture atlas list is wrong. As a result:

• The list contains textures that don't exist. Open the texture picker, type 2x in the search field. You will see a bunch of empty textures among the results. Let's take the first one: !ui-frame-diamondmetal-edgeleft-2x. /dump C_Texture.GetAtlasInfo("!ui-frame-diamondmetal-edgeleft-2x") retuns nil
• Some existing textures are missing from the list. For example: Unit_Druid_AstralPower_Fill or Unit_DemonHunter_Fury_Fill. Searching in the texture picker turns up nothing. /dump C_Texture.GetAtlasInfo("Unit_Druid_AstralPower_Fill") retuns a non-empty result.

The whole point is that the required data is not in CommitedName column of the UiTextureAtlasMember table but in Name column of the UiTextureAtlasElement table. However, you can't just list the values of this column. That would also be a mistake. To get the correct result, you will have to use data from all three tables (UiTextureAtlas, UiTextureAtlasMember, UiTextureAtlasElement) by correlating their elements based on matching keys. Basically, it's an inner join operation in relational databases.

The algorithm might be something like this:

1. Iterate over all records in the UiTextureAtlas table. Follow step 2 for each item.
2. Find in the UiTextureAtlasMember table all items for which UiTextureAtlasID is equal to ID of the item obtained in step 1. Follow step 3 for each such item.
3. Find in the UiTextureAtlasElement table all items for which ID is equal to UiTextureAtlasElementID of the item obtained in step 2. For each such item, add value of the Name column to the result if it's not already there

But that's not all. When generating the list in this way, it was found that:

• There is a group of atlases whose names begin with CGuy_. C_Texture.GetAtlasInfo returns info for them, but they are displayed as a green square on the screen (check it out using the texture picker). I can assume by the file name (Interface/CameraGuy/CameraGuyAsset) that they are intended for the commentator mode, which is not available to players
• The tables may contain some "special" records. These are either records that have apparently been forgotten to be deleted or that have been made for future changes, or records related to the store. Maybe that's something else. Attempts to find files related to them, for example on Wago, end in nothing. Often such textures are displayed as empty.

So I decided to further filter the result by checking whether a texture atlas file (FileDataID column of the UiTextureAtlas table) exists in the game client. I used listfile of the current build for this purpose.

I've also used the following tools:

• CASCExplorer - extract databases and listfile for the current build
• DBC2CSV - convert DB2 to CSV
• wow-listfile - the latest community listfile for CASCExplorer
• WoWDBDefs - the latest database field definitions for DBC2CSV

Note: Personally, I didn't use compiled binaries of CASCExplorer and DBC2CSV, but built them myself as they may have become outdated since the last release.

Notes: I can't say that the list in the commit below is complete, because there is no sample/benchmark to compare with. In theory, the list doesn't contain empty textures.

Data are extracted from the following builds:

• Retail - 10.2.0.52649
• WOTLK - 3.4.3.52237
• Classic - 1.15.0.52610

Type of change

• Bug fix (non-breaking change which fixes an issue)

[InfusOnWoW]

I would prefer if we can automate this in the same way we used to automate the model list update.

If I understood you correctly, we can simplify the algorithm and do:

• Iterate over all records in the UiTextureAtlas table, add all IDs to a validTextureAtlasId table
• Iterate over all records in UiTextureAtlasMember table for all items for which UiTextureAtlasID is contained in validTextureAtlasId, add the id to validAtlasElementId
• Iterate UiTextureAtlasElement table for all items for which ID is contained in validAtalsElementID, add value of the Name column to the result.

I'll take a look at whether we can automate this with wago.tools in the new year.

[mrbuds]

As explained at #3966 this export is generated with wow.export to get db2 files, into dbc2csv, into csv2lua, basically same as you did but not using cascexplorer

Can you include the atlascsv2lua.lua script with your changes?

[Nightwarden24]

As explained at #3966 this export is generated with wow.export to get db2 files, into dbc2csv, into csv2lua, basically same as you did but not using cascexplorer

Can you include the atlascsv2lua.lua script with your changes?

I used C#. If necessary I will attach the entire project. The above algorithm fits in just one line

//Method syntax
List<string> preResult = filteredTableL.Join(tableA, itemF => itemF.FileDataID, itemA => itemA.FileDataID, (itemF, itemA) => itemA)
                                       .Join(tableM, prev => prev.ID, itemM => itemM.UiTextureAtlasID, (prev, itemM) => itemM)
                                       .Join(tableE, prec => prec.UiTextureAtlasElementID, itemE => itemE.ID, (prec, itemE) => itemE.Name)
                                       .Distinct(StringComparer.Ordinal)
                                       .OrderBy(x => x, StringComparer.Ordinal)
                                       .ToList();

where

• filteredTableL - filtered listfile (only textures) for the current build
• tableA - UiTextureAtlas
• tableM - UiTextureAtlasMember
• tableE - UiTextureAtlasElement
• preResult - pre-result, next I check for the presence of invalid characters in atlas name (', ", space)

[Nightwarden24]

The same but in query syntax with one note

//Since there is no equivalent to Distinct() in queries
//Use   group A by B into C
//      select C.First()

//Query syntax
var query = from itemF in filteredTableL
           join itemA in tableA on itemF.FileDataID equals itemA.FileDataID
           join itemM in tableM on itemA.ID equals itemM.UiTextureAtlasID
           join itemE in tableE on itemM.UiTextureAtlasElementID equals itemE.ID
           orderby itemE.Name ascending
           group itemE by itemE.Name into newForm
           select newForm.First().Name;
List<string> preResult = query.ToList();

[Nightwarden24]

I would prefer if we can automate this in the same way we used to automate the model list update.

If I understood you correctly, we can simplify the algorithm and do:

• Iterate over all records in the UiTextureAtlas table, add all IDs to a validTextureAtlasId table
• Iterate over all records in UiTextureAtlasMember table for all items for which UiTextureAtlasID is contained in validTextureAtlasId, add the id to validAtlasElementId
• Iterate UiTextureAtlasElement table for all items for which ID is contained in validAtalsElementID, add value of the Name column to the result.

I'll take a look at whether we can automate this with wago.tools in the new year.

I'm not quite sure what your main point is. Would you like to automate or/and use Github Actions for this task? Ok. There are no problems with this at all, except for data extraction. More precisely, getting one type of data. I duplicate one thought from the above text wall. The list generated using the tables may contain empty textures (Why? Blizzard knows it). How to get rid of them? I don't know. You can suggest your option. I used listfile for the current build (NOT community listfile, it's important). The problem is getting this file. At the moment I don’t know any other tools for this except CASCExplorer and Local WoW.tools

[InfusOnWoW]

Thank you for all the work you did on this. I've merged the lua implementation of you algorighm. If you can get the CascLib to work, I can look into integrating that too.