Skip to content

Commit 53c07c9

Browse files
Improve Analyze documentation, small code fixes (#25)
Add a new topic to the documentation that gives some example usage of Analyze, for example to run queries on the command line. Rework the README.md for Analyze library to move the more advanced developer focused information to the bottom. (I think most readers will just want to understand what Analyze does, and only more rare cases will they want to understand the implementation and actually extend library ) Add more details about how to extend the Analyze library to add support for more types. Trivial code changes: Remove dead code - SerializedObject base class is not used. Fix two warnings about unreferenced variables.
1 parent 88285a7 commit 53c07c9

6 files changed

Lines changed: 294 additions & 52 deletions

File tree

Analyzer/PPtrAndCrcProcessor.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ private UnityFileReader GetResourceReader(string filename)
6565
{
6666
reader = new UnityFileReader(Path.Join(m_Folder, filename), 4 * 1024 * 1024);
6767
}
68-
catch (Exception e)
68+
catch (Exception)
6969
{
7070
Console.Error.WriteLine();
7171
Console.Error.WriteLine($"Error opening resource file {filename}");

Analyzer/README.md

Lines changed: 42 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -4,37 +4,20 @@ The Analyzer is a class library that can be used to analyze the content of Unity
44
as AssetBundles and SerializedFiles. It iterates through all the serialized objects and uses the
55
TypeTree to extract information about these objects (e.g. name, size, etc.)
66

7-
The extracted information is forwarded to an object implementing the [IWriter](./IWriter.cs)
8-
interface. The library provides the [SQLiteWriter](./SQLite/SQLiteWriter.cs) implementation that
9-
writes the data into a SQLite database.
7+
The most common use of this library is through the [analyze](../UnityDataTool/README.md#analyzeanalyse)
8+
command of the UnityDataTool. This uses the Analyze library to generate a SQLite database.
109

11-
It is possible to extract type-specific properties using classes inheriting from the
12-
[SerializedObject](./SerializedObjects/SerializedObject.cs) class. The project already provides
13-
the most common SerializedObject implementations in the [SerializedObjects](./SerializedObjects)
14-
folder.
10+
Once generated, a tool such as the [DB Browser for SQLite](https://sqlitebrowser.org/), or the command line `sqlite3` tool, can be used to look at the content of the database.
1511

16-
# How to use the library
12+
# Example usage
1713

18-
The [AnalyzerTool](./AnalyzerTool.cs) class is the API entry point. The main method is called
19-
Analyze. It is currently hardcoded to write using the [SQLiteWriter](./SQLite/SQLiteWriter.cs),
20-
but it can easily be adapter to use another writer type. It takes four parameters:
21-
* path (string): path of the folder where the files to analyze are located. It will be searched
22-
recursively.
23-
* databaseName (string): database filename, it will be overwritten if it already exists.
24-
* searchPattern (string): file search pattern (e.g. \*.bundle).
25-
* skipReferences (bool): determines if the CRC calculation and references (PPtrs) extraction must
26-
skipped. This is faster, but the refs table will be empty and the duplicate assets won't be
27-
accurate.
28-
Calling this method will create the SQLite output database and will recursively
29-
process the files matching the search pattern in the provided path. It will add a row in
30-
the 'objects' table for each serialized object. This table contain basic information such as the
31-
size and the name of the object (if it has one).
32-
33-
# How to use the database
34-
35-
A tool such as the [DB Browser for SQLite](https://sqlitebrowser.org/) is required to look at the
36-
content of the database. The database provides different views that can be used to easily find the
37-
information you might need.
14+
See [this topic](../Documentation/analyze-examples.md) for examples of how to use the SQLite output of the UnityDataTool Analyze command.
15+
16+
# DataBase Reference
17+
18+
The database provides different views. The views join multiple tables together and often it is not necessary to write your own SQL queries to find the information you want, especially when you are using a visual SQLite tool.
19+
20+
This section gives an overview of the main views.
3821

3922
## object_view
4023

@@ -178,3 +161,34 @@ stripping it won't make a big difference.
178161
This view lists all the shaders aggregated by name. The *instances* column indicates how many time
179162
the shader was found in the data files. It also provides the total size per shader and the list of
180163
AssetBundles in which they were found.
164+
165+
# Advanced
166+
167+
## Using the library
168+
169+
The [AnalyzerTool](./AnalyzerTool.cs) class is the API entry point. The main method is called
170+
Analyze. It is currently hard coded to write using the [SQLiteWriter](./SQLite/SQLiteWriter.cs),
171+
but this approach could be extended to add support for other outputs.
172+
173+
Calling this method will recursively process the files matching the search pattern in the provided
174+
path. It will add a row in the 'objects' table for each serialized object. This table contain basic
175+
information such as the size and the name of the object (if it has one).
176+
177+
## Extending the Library
178+
179+
The extracted information is forwarded to an object implementing the [IWriter](./IWriter.cs)
180+
interface. The library provides the [SQLiteWriter](./SQLite/SQLiteWriter.cs) implementation that
181+
writes the data into a SQLite database.
182+
183+
The core properties that apply to all Unity Objects are extracted into the `objects` table.
184+
However much of the most useful Analyze functionality comes by virtue of the type-specific information that is extracted for
185+
important types like Meshes, Shaders, Texture2D and AnimationClips. For example, when a Mesh object is encountered in a Serialized
186+
File, then rows are added to both the `objects` table and the `meshes` table. The meshes table contains columns that only apply to Mesh objects, for example the number of vertices, indices, bones, and channels. The `mesh_view` is a view that joins the `objects` table with the `meshes` table, so that you can see all the properties of a Mesh object in one place.
187+
188+
Each supported Unity object type follows the same pattern:
189+
* A Handler class in the SQLite/Handlers (e.g. [MeshHandler.cs](./SQLite/Handler/MeshHandler.cs).
190+
* The registration of the handler in the m_Handlers dictionary in [SQLiteWriter.cs](./SQLite/SQLiteWriter.cs).
191+
* SQL statements defining extra tables and views associated with the type, e.g. [Mesh.sql](./SQLite/Resources/Mesh.sql).
192+
* A Reader class that uses RandomAccessReader to read properties from the serialized object. e.g. [Mesh.cs](./SerializedObjects/Mesh.cs).
193+
194+
It would be possible to extend the Analyze library to add additional columns for the existing types, or by following the same pattern to add additional types. The [dump](../UnityDataTool/README.md#dump) feature of UnityDataTool is a useful way to see the property names and other details of the serialization for a type. Based on that information, code in the Reader class can use the RandomAccessReader to retrieve those properties to bring them into the SQLite database.

Analyzer/SQLite/SQLiteWriter.cs

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ public class SQLiteWriter : IWriter
2323
private Util.ObjectIdProvider m_ObjectIdProvider = new ();
2424

2525
private Regex m_RegexSceneFile = new(@"BuildPlayer-([^\.]+)(?:\.sharedAssets)?");
26-
26+
2727
// Used to map PPtr fileId to its corresponding serialized file id in the database.
2828
Dictionary<int, int> m_LocalToDbFileId = new ();
2929

@@ -37,7 +37,7 @@ public class SQLiteWriter : IWriter
3737
{ "AssetBundle", new AssetBundleHandler() },
3838
{ "PreloadData", new PreloadDataHandler() },
3939
};
40-
40+
4141
private SqliteConnection m_Database;
4242
private SqliteCommand m_AddReferenceCommand = new SqliteCommand();
4343
private SqliteCommand m_AddAssetBundleCommand = new SqliteCommand();
@@ -82,9 +82,9 @@ public void Begin()
8282
// Console.WriteLine($"Connection state before init: {m_Database.State}");
8383
handler.Init(m_Database);
8484
// Console.WriteLine($"Connection state after init: {m_Database.State}");
85-
85+
8686
}
87-
87+
8888
CreateSQLiteCommands();
8989
}
9090

@@ -94,7 +94,7 @@ public void End()
9494
{
9595
throw new InvalidOperationException("SQLiteWriter.End called before SQLiteWriter.Begin");
9696
}
97-
97+
9898
foreach (var handler in m_Handlers.Values)
9999
{
100100
handler.Finalize(m_Database);
@@ -125,7 +125,7 @@ private void CreateSQLiteCommands()
125125
m_AddReferenceCommand.Parameters.Add("@referenced_object", SqliteType.Integer);
126126
m_AddReferenceCommand.Parameters.Add("@property_path", SqliteType.Text);
127127
m_AddReferenceCommand.Parameters.Add("@property_type", SqliteType.Text);
128-
128+
129129
m_AddObjectCommand = m_Database.CreateCommand();
130130
m_AddObjectCommand.CommandText = "INSERT INTO objects (id, object_id, serialized_file, type, name, game_object, size, crc32) VALUES (@id, @object_id, @serialized_file, @type, @name, @game_object, @size, @crc32)";
131131
m_AddObjectCommand.Parameters.Add("@id", SqliteType.Integer);
@@ -154,7 +154,7 @@ public void BeginAssetBundle(string name, long size)
154154
{
155155
throw new InvalidOperationException("SQLWriter.BeginAssetBundle called twice");
156156
}
157-
157+
158158
m_CurrentAssetBundleId = m_NextAssetBundleId++;
159159
m_AddAssetBundleCommand.Parameters["@id"].Value = m_CurrentAssetBundleId;
160160
m_AddAssetBundleCommand.Parameters["@name"].Value = name;
@@ -171,7 +171,7 @@ public void EndAssetBundle()
171171

172172
m_CurrentAssetBundleId = -1;
173173
}
174-
174+
175175
public void WriteSerializedFile(string relativePath, string fullPath, string containingFolder)
176176
{
177177
using var sf = UnityFileSystem.OpenSerializedFile(fullPath);
@@ -188,8 +188,8 @@ public void WriteSerializedFile(string relativePath, string fullPath, string con
188188
if (match.Success)
189189
{
190190
var sceneName = match.Groups[1].Value;
191-
192-
// There is no Scene object in Unity (a Scene is the full content of a
191+
192+
// There is no Scene object in Unity (a Scene is the full content of a
193193
// SerializedFile). We generate an object id using the name of the Scene
194194
// as SerializedFile name, and the object id 0.
195195
sceneId = m_ObjectIdProvider.GetId((m_SerializedFileIdProvider.GetId(sceneName), 0));
@@ -212,7 +212,7 @@ public void WriteSerializedFile(string relativePath, string fullPath, string con
212212
m_AddObjectCommand.ExecuteNonQuery();
213213
}
214214
}
215-
215+
216216
m_LocalToDbFileId.Clear();
217217

218218
Context ctx = new()
@@ -275,7 +275,7 @@ public void WriteSerializedFile(string relativePath, string fullPath, string con
275275
{
276276
name = randomAccessReader["m_Name"].GetValue<string>();
277277
}
278-
278+
279279
if (randomAccessReader.HasChild("m_GameObject"))
280280
{
281281
var pptr = randomAccessReader["m_GameObject"];
@@ -316,7 +316,7 @@ public void WriteSerializedFile(string relativePath, string fullPath, string con
316316

317317
transaction.Commit();
318318
}
319-
catch (Exception e)
319+
catch (Exception)
320320
{
321321
transaction.Rollback();
322322
throw;

Analyzer/SerializedObjects/SerializedObject.cs

Lines changed: 0 additions & 9 deletions
This file was deleted.

0 commit comments

Comments
 (0)