diff --git a/README.md b/README.md index 604c65e..2aba007 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,7 @@ questions, reference answers, reports and conversion live in swift run -c release FluidUseLaya answer --state "…" --type choice \ --instructions "What does the customer want?" --options "refund|order status|technical help" swift run -c release FluidUseLaya tetris --shortlist --describe graded --pieces 200 # headless Tetris, P(clean) per landing -swift run -c release FluidUseLaya 2048 --model-dir /path/to/gliclass --precision lut8 --games 10 +swift run -c release FluidUseLaya 2048 --precision lut8 --games 10 swift run -c release FluidUseLaya benchmark --suites /benchmark/suites.jsonl --reference /benchmark/reference-rows.jsonl swift run -c release LayaTetrisDemo # SwiftUI: GLiClass/laya play Tetris swift run -c release GLiClass2048Demo # SwiftUI: GLiClass plays 2048 @@ -121,8 +121,15 @@ The on-device models, measured on the same Mac with checked-in reports: [Benchma CUA-S1-FORMS: 0.9 ms per decision on the Neural Engine, accuracy identical to PyTorch on the 24,370-row synthetic test. laya: 3.6 ms per short question, identical to PyTorch on laya's ten published suites, e8 buckets 30% smaller at the same accuracy. GLiClass Edge Apps v2: 1.61 ms FP16 -or 1.81 ms LUT8 for a two-option L128 decision, with its conversion pipeline in +or 1.81 ms LUT8 for a two-option L128 decision. Its +[Core ML packages and config](https://huggingface.co/FluidInference/gliclass-edge-apps-coreml) +are on Hugging Face, with the conversion pipeline in [mobius PR #101](https://github.com/FluidInference/mobius/pull/101). +GLiClass demos download the selected Core ML bucket and tokenizer on first use. The loader reads the +published `config.json`, checks each file against `checksums.json`, and caches the assets under +`~/Library/Application Support/FluidUse/Models/gliclass-edge-apps-coreml`. `GLICLASS_MODEL_DIR` +or CLI `--model-dir` still selects a local directory. The Hub publishes FP16 at L128/L256/L512 and +LUT8 at L128; other local precision variants require an explicit local directory. ## Scope diff --git a/Sources/Decision2048BenchDemo/Decision2048BenchModel.swift b/Sources/Decision2048BenchDemo/Decision2048BenchModel.swift index 04e6965..e1ccd0e 100644 --- a/Sources/Decision2048BenchDemo/Decision2048BenchModel.swift +++ b/Sources/Decision2048BenchDemo/Decision2048BenchModel.swift @@ -94,13 +94,15 @@ final class Decision2048BenchModel: ObservableObject { Task { do { let environment = ProcessInfo.processInfo.environment - guard let gliClassDirectory = environment["GLICLASS_MODEL_DIR"], !gliClassDirectory.isEmpty else { - throw GLiClassError.invalidAsset("Set GLICLASS_MODEL_DIR for the 2048 benchmark") + let gliClassConfiguration = GLiClassManager.Configuration( + lengths: [128], precision: environment["GLICLASS_PRECISION"] ?? "lut8") + let loadedGLiClass: GLiClassManager + if let directory = environment["GLICLASS_MODEL_DIR"], !directory.isEmpty { + loadedGLiClass = try await GLiClassManager.load( + from: URL(fileURLWithPath: directory), configuration: gliClassConfiguration) + } else { + loadedGLiClass = try await GLiClassManager.load(configuration: gliClassConfiguration) } - let loadedGLiClass = try await GLiClassManager.load( - from: URL(fileURLWithPath: gliClassDirectory), - configuration: .init( - lengths: [128], precision: environment["GLICLASS_PRECISION"] ?? "lut8")) loadStatus = "Loading Laya E8…" let layaConfiguration = LayaManager.Configuration( lengths: [128], precision: environment["LAYA_PRECISION"] ?? "e8") diff --git a/Sources/Decision2048BenchDemo/README.md b/Sources/Decision2048BenchDemo/README.md index 98a3ff7..01750c0 100644 --- a/Sources/Decision2048BenchDemo/README.md +++ b/Sources/Decision2048BenchDemo/README.md @@ -6,11 +6,12 @@ expectimax shortlist to each model. GLiClass compares the candidates in one clas its established `noul` question to each candidate, requiring two passes per move. ```bash -GLICLASS_MODEL_DIR=/path/to/gliclass-assets \ -LAYA_MODEL_DIR="$HOME/Library/Application Support/FluidUse/Models/laya-coreml" \ swift run -c release Decision2048BenchDemo ``` +Both models download their published Core ML assets on first use. To use existing assets, set +`GLICLASS_MODEL_DIR` and/or `LAYA_MODEL_DIR` to their respective model directories. + Set `GAME2048_AUTOLOAD=1 GAME2048_AUTORUN=1` to launch immediately, `GAME2048_SEED=` to select the deterministic starting seed, and `GAME2048_DELAY_MS=` to control visual pacing. Model inference runs alternately so the models do not contend for the Neural Engine; the visual delay is excluded from latency. diff --git a/Sources/FluidUse/GLiClass/GLiClassManager.swift b/Sources/FluidUse/GLiClass/GLiClassManager.swift index af4435c..1137778 100644 --- a/Sources/FluidUse/GLiClass/GLiClassManager.swift +++ b/Sources/FluidUse/GLiClass/GLiClassManager.swift @@ -75,6 +75,19 @@ public actor GLiClassManager { self.tokenizer = tokenizer } + /// Download requested buckets from the Hub into the FluidUse model cache and load them. + /// - Parameter cacheDirectory: The parent Models directory, not the repository subdirectory. + public static func load( + cacheDirectory: URL? = nil, + configuration: Configuration = Configuration(), + progress: GLiClassModelStore.Progress? = nil + ) async throws -> GLiClassManager { + let directory = try await GLiClassModelStore.ensure( + lengths: configuration.lengths, precision: configuration.precision, + cacheDirectory: cacheDirectory, progress: progress) + return try await load(from: directory, configuration: configuration) + } + /// Load `.mlmodelc`/`.mlpackage` buckets and `tokenizer.json` from a local directory. public static func load( from directory: URL, configuration: Configuration = Configuration() diff --git a/Sources/FluidUse/GLiClass/GLiClassModelStore.swift b/Sources/FluidUse/GLiClass/GLiClassModelStore.swift new file mode 100644 index 0000000..9f3ea65 --- /dev/null +++ b/Sources/FluidUse/GLiClass/GLiClassModelStore.swift @@ -0,0 +1,154 @@ +import CryptoKit +import Foundation + +/// Downloads the GLiClass Core ML packages described by the Hub repository's `config.json`. +public enum GLiClassModelStore { + public static let repository = "FluidInference/gliclass-edge-apps-coreml" + public typealias Progress = @Sendable (_ file: String, _ bytes: Int64) -> Void + + static let packageMembers = [ + "Manifest.json", "Data/com.apple.CoreML/model.mlmodel", "Data/com.apple.CoreML/weights/weight.bin", + ] + + struct RepositoryConfig: Decodable { + struct Bucket: Decodable { + let length: Int + let fp16: String? + let lut8: String? + } + + let format: String + let maxOptions: Int + let tokenizer: String + let buckets: [Bucket] + + enum CodingKeys: String, CodingKey { + case format, tokenizer, buckets + case maxOptions = "max_options" + } + + func package(length: Int, precision: String) throws -> String { + guard format == "coreml", maxOptions == GLiClassManager.maximumOptions, + tokenizer == "tokenizer.json" + else { throw GLiClassError.invalidAsset("Unexpected GLiClass repository config") } + guard let bucket = buckets.first(where: { $0.length == length }) else { + throw GLiClassError.invalidAsset( + "GLiClass L\(length) is not published in \(GLiClassModelStore.repository)") + } + let package: String? + switch precision { + case "fp16": package = bucket.fp16 + case "lut8": package = bucket.lut8 + default: package = nil + } + guard let package, + package == (try GLiClassManager.modelName(length: length, precision: precision)) + ".mlpackage" + else { + throw GLiClassError.invalidAsset( + "GLiClass \(precision) L\(length) is not published in \(GLiClassModelStore.repository)") + } + return package + } + } + + /// Ensure selected packages and tokenizer exist under `cacheDirectory/gliclass-edge-apps-coreml`. + /// The Hub config selects the artifacts; published SHA-256 hashes validate cached and downloaded files. + public static func ensure( + lengths: [Int], precision: String = "fp16", cacheDirectory: URL? = nil, progress: Progress? = nil + ) async throws -> URL { + guard !lengths.isEmpty else { throw GLiClassError.invalidAsset("At least one GLiClass bucket is required") } + let root = cacheDirectory ?? LayaModelStore.defaultCacheDirectory() + let directory = root.appendingPathComponent("gliclass-edge-apps-coreml", isDirectory: true) + let configData = try await downloadData("config.json") + let checksumsData = try await downloadData("checksums.json") + let config = try JSONDecoder().decode(RepositoryConfig.self, from: configData) + let checksums = try JSONDecoder().decode([String: String].self, from: checksumsData) + guard let configHash = checksums["config.json"], sha256(configData) == configHash else { + throw GLiClassError.invalidAsset("GLiClass config.json checksum mismatch") + } + var paths = [config.tokenizer] + for length in lengths { + let package = try config.package(length: length, precision: precision) + paths += packageMembers.map { "\(package)/\($0)" } + } + let manager = FileManager.default + try manager.createDirectory(at: directory, withIntermediateDirectories: true) + try configData.write(to: directory.appendingPathComponent("config.json"), options: .atomic) + try checksumsData.write(to: directory.appendingPathComponent("checksums.json"), options: .atomic) + for relative in paths { + guard let expectedHash = checksums[relative], expectedHash.count == 64 else { + throw GLiClassError.invalidAsset("No checksum for \(relative)") + } + let destination = directory.appendingPathComponent(relative) + if manager.fileExists(atPath: destination.path), try sha256(file: destination) == expectedHash { + continue + } + try manager.createDirectory(at: destination.deletingLastPathComponent(), withIntermediateDirectories: true) + progress?(relative, 0) + let temporary = try await downloadFile(relative) + defer { try? manager.removeItem(at: temporary) } + guard try sha256(file: temporary) == expectedHash else { + throw GLiClassError.invalidAsset("GLiClass checksum mismatch for \(relative)") + } + let size = (try manager.attributesOfItem(atPath: temporary.path)[.size] as? NSNumber)?.int64Value ?? 0 + try LayaModelStore.installDownloadedFile(temporary, at: destination) + progress?(relative, size) + } + return directory + } + + private static func url(for relative: String) throws -> URL { + let encoded = relative.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed) ?? relative + guard let url = URL(string: "https://huggingface.co/\(repository)/resolve/main/\(encoded)") else { + throw GLiClassError.invalidAsset("Bad GLiClass download URL for \(relative)") + } + return url + } + + private static func downloadData(_ relative: String) async throws -> Data { + let (data, response) = try await URLSession.shared.data(from: url(for: relative)) + try validate(response, file: relative, size: Int64(data.count)) + return data + } + + private static func downloadFile(_ relative: String) async throws -> URL { + let (temporary, response) = try await URLSession.shared.download(from: url(for: relative)) + do { + let size = + (try FileManager.default.attributesOfItem(atPath: temporary.path)[.size] as? NSNumber)? + .int64Value ?? 0 + try validate(response, file: relative, size: size) + return temporary + } catch { + try? FileManager.default.removeItem(at: temporary) + throw error + } + } + + private static func validate(_ response: URLResponse, file: String, size: Int64) throws { + guard let http = response as? HTTPURLResponse, (200..<300).contains(http.statusCode) else { + throw GLiClassError.invalidAsset( + "Download of \(file) failed (\((response as? HTTPURLResponse)?.statusCode ?? -1))") + } + guard !((http.value(forHTTPHeaderField: "Content-Type") ?? "").contains("text/html")) else { + throw GLiClassError.invalidAsset("Download of \(file) returned HTML") + } + guard size > 0, http.expectedContentLength <= 0 || size == http.expectedContentLength else { + throw GLiClassError.invalidAsset("Download of \(file) has an unexpected size") + } + } + + static func sha256(_ data: Data) -> String { + SHA256.hash(data: data).map { String(format: "%02x", $0) }.joined() + } + + private static func sha256(file: URL) throws -> String { + let handle = try FileHandle(forReadingFrom: file) + defer { try? handle.close() } + var digest = SHA256() + while let chunk = try handle.read(upToCount: 1_048_576), !chunk.isEmpty { + digest.update(data: chunk) + } + return digest.finalize().map { String(format: "%02x", $0) }.joined() + } +} diff --git a/Sources/FluidUseLaya/Game2048Command.swift b/Sources/FluidUseLaya/Game2048Command.swift index 6ddcd3a..057c5a4 100644 --- a/Sources/FluidUseLaya/Game2048Command.swift +++ b/Sources/FluidUseLaya/Game2048Command.swift @@ -76,12 +76,13 @@ struct Game2048Command { let options = try parse(arguments) let manager: GLiClassManager? if options.policy == "gliclass" { - guard let modelDirectory = options.modelDirectory else { - throw GLiClassError.invalidAsset("--policy gliclass requires --model-dir") + let configuration = GLiClassManager.Configuration(lengths: [128], precision: options.precision) + if let modelDirectory = options.modelDirectory { + manager = try await GLiClassManager.load( + from: URL(fileURLWithPath: modelDirectory), configuration: configuration) + } else { + manager = try await GLiClassManager.load(configuration: configuration) } - manager = try await GLiClassManager.load( - from: URL(fileURLWithPath: modelDirectory), - configuration: .init(lengths: [128], precision: options.precision)) } else { manager = nil } diff --git a/Sources/FluidUseLaya/LayaTetrisCommand.swift b/Sources/FluidUseLaya/LayaTetrisCommand.swift index 59c54bb..da62418 100644 --- a/Sources/FluidUseLaya/LayaTetrisCommand.swift +++ b/Sources/FluidUseLaya/LayaTetrisCommand.swift @@ -123,12 +123,13 @@ struct LayaTetrisCommand { layaManager = try await LayaManager.load(configuration: configuration) } } else if options.policy == "gliclass" { - guard let directory = options.modelDirectory else { - throw GLiClassError.invalidAsset("--policy gliclass currently requires --model-dir") + let configuration = GLiClassManager.Configuration(lengths: options.lengths, precision: options.precision) + if let directory = options.modelDirectory { + gliClassManager = try await GLiClassManager.load( + from: URL(fileURLWithPath: directory), configuration: configuration) + } else { + gliClassManager = try await GLiClassManager.load(configuration: configuration) } - gliClassManager = try await GLiClassManager.load( - from: URL(fileURLWithPath: directory), - configuration: .init(lengths: options.lengths, precision: options.precision)) } let gliClassLabels = [ "a poor Tetris placement that creates holes or a dangerous tall stack", @@ -369,8 +370,9 @@ struct LayaTetrisCommand { Plays headless 10x20 Tetris. With --policy laya (default) every legal landing is described in one sentence and scored by laya's P(clean); the best-scoring landing is played. - With --policy gliclass, --model-dir must hold tokenizer.json and the GLiClass Edge Apps v2 - Core ML bucket. The same candidates and descriptions are scored for an apples-to-apples game. + With --policy gliclass, the published GLiClass bucket downloads on first use. --model-dir + can instead point to local tokenizer.json and Core ML bucket assets. The same candidates + and descriptions are scored for an apples-to-apples game. --gliclass-choice compare candidate descriptions in one encoder pass --gliclass-candidates N heuristic prefilter width for choice mode (default 2; max 25) --gliclass-margin P minimum probability margin before GLiClass overrides the heuristic leader diff --git a/Sources/GLiClass2048Demo/Game2048Model.swift b/Sources/GLiClass2048Demo/Game2048Model.swift index 8ee6de9..49db934 100644 --- a/Sources/GLiClass2048Demo/Game2048Model.swift +++ b/Sources/GLiClass2048Demo/Game2048Model.swift @@ -92,13 +92,17 @@ final class Game2048Model: ObservableObject { loadStatus = "Loading GLiClass Edge Apps v2…" Task { do { - guard let directory = ProcessInfo.processInfo.environment["GLICLASS_MODEL_DIR"], !directory.isEmpty - else { throw GLiClassError.invalidAsset("Set GLICLASS_MODEL_DIR for the 2048 demo") } - let precision = ProcessInfo.processInfo.environment["GLICLASS_PRECISION"] ?? "lut8" + let environment = ProcessInfo.processInfo.environment + let precision = environment["GLICLASS_PRECISION"] ?? "lut8" let started = Date() - let loaded = try await GLiClassManager.load( - from: URL(fileURLWithPath: directory), - configuration: .init(lengths: [128], precision: precision)) + let configuration = GLiClassManager.Configuration(lengths: [128], precision: precision) + let loaded: GLiClassManager + if let directory = environment["GLICLASS_MODEL_DIR"], !directory.isEmpty { + loaded = try await GLiClassManager.load( + from: URL(fileURLWithPath: directory), configuration: configuration) + } else { + loaded = try await GLiClassManager.load(configuration: configuration) + } _ = try await loaded.classify( text: "Build the largest tile without filling the board.", labels: ["swipe left: 8 empty cells", "swipe right: 5 empty cells"], diff --git a/Sources/GLiClass2048Demo/README.md b/Sources/GLiClass2048Demo/README.md index 37234d3..8d86a28 100644 --- a/Sources/GLiClass2048Demo/README.md +++ b/Sources/GLiClass2048Demo/README.md @@ -5,9 +5,14 @@ heuristic shortlists the strongest legal swipes; GLiClass compares their resulti in one L128 encoder pass. ```bash -GLICLASS_MODEL_DIR=/path/to/assets GLICLASS_PRECISION=lut8 swift run -c release GLiClass2048Demo +swift run -c release GLiClass2048Demo ``` +The default L128 LUT8 package and tokenizer download from +[FluidInference/gliclass-edge-apps-coreml](https://huggingface.co/FluidInference/gliclass-edge-apps-coreml) +on first use and are cached locally. Set `GLICLASS_MODEL_DIR=/path/to/assets` to use local assets, +or `GLICLASS_PRECISION=fp16` for the published FP16 bucket. + Set `GAME2048_AUTORUN=1` to load and play immediately, `GAME2048_SEED=` to choose the deterministic tile sequence, `GAME2048_CANDIDATES=2|3|4` to choose the comparison width, and `GAME2048_MARGIN=0...1` to require a confidence margin before GLiClass overrides the heuristic leader. diff --git a/Sources/LayaTetrisDemo/GameModel.swift b/Sources/LayaTetrisDemo/GameModel.swift index a2d1615..c4fceb6 100644 --- a/Sources/LayaTetrisDemo/GameModel.swift +++ b/Sources/LayaTetrisDemo/GameModel.swift @@ -163,13 +163,16 @@ final class GameModel: ObservableObject { let started = Date() switch policy { case .gliclass: - guard let directory = ProcessInfo.processInfo.environment["GLICLASS_MODEL_DIR"], - !directory.isEmpty - else { throw GLiClassError.invalidAsset("Set GLICLASS_MODEL_DIR for the GLiClass demo") } - let precision = ProcessInfo.processInfo.environment["GLICLASS_PRECISION"] ?? "fp16" - let loaded = try await GLiClassManager.load( - from: URL(fileURLWithPath: directory), - configuration: .init(lengths: [128], precision: precision)) + let environment = ProcessInfo.processInfo.environment + let precision = environment["GLICLASS_PRECISION"] ?? "fp16" + let configuration = GLiClassManager.Configuration(lengths: [128], precision: precision) + let loaded: GLiClassManager + if let directory = environment["GLICLASS_MODEL_DIR"], !directory.isEmpty { + loaded = try await GLiClassManager.load( + from: URL(fileURLWithPath: directory), configuration: configuration) + } else { + loaded = try await GLiClassManager.load(configuration: configuration) + } _ = try await loaded.classify( text: "The piece buries nothing and keeps the stack low.", labels: ["a poor Tetris placement", "a clean Tetris placement"], diff --git a/Sources/LayaTetrisDemo/README.md b/Sources/LayaTetrisDemo/README.md index 9d1891d..897eae8 100644 --- a/Sources/LayaTetrisDemo/README.md +++ b/Sources/LayaTetrisDemo/README.md @@ -9,9 +9,10 @@ milliseconds per move, and calls. swift run -c release LayaTetrisDemo ``` -No Xcode project is needed. Set `GLICLASS_MODEL_DIR` to a directory holding `tokenizer.json` and -`gliclass_edge_apps_fp16_L128_options25.mlpackage`, or select laya and let it download from -`FluidInference/laya-coreml` (`LAYA_MODEL_DIR` skips that download). **Play** scores flat out with +No Xcode project is needed. GLiClass downloads its L128 FP16 package and tokenizer from +`FluidInference/gliclass-edge-apps-coreml` on first use. Set `GLICLASS_MODEL_DIR` to a local asset +directory to skip that download. Laya downloads from `FluidInference/laya-coreml` unless +`LAYA_MODEL_DIR` selects a local directory. **Play** scores flat out with Core ML on CPU + ANE and continues on the next seed after a top-out. SwiftUI presents the evolving board at the display refresh rate while the model stays hot. The *pause per piece* slider can expose individual moves, but long pauses let the Neural Engine idle and raise measured latency. The *delay per scored @@ -19,8 +20,8 @@ landing* slider slows laya's candidate-by-candidate scoring so it can be watched outline); the chosen landing is drawn in green. The GLiClass training, Core ML conversion, compression, and parity reports are in -[mobius PR #101](https://github.com/FluidInference/mobius/pull/101). The GLiClass packages are not yet -auto-downloaded, so build or obtain those assets before running this policy. +[mobius PR #101](https://github.com/FluidInference/mobius/pull/101). The published GLiClass packages +and runtime config are at [Hugging Face](https://huggingface.co/FluidInference/gliclass-edge-apps-coreml). **Policy** switches among GLiClass, laya, a feature-weighted heuristic, and random play. The harness toggle applies to every policy. See [Benchmarks.md](../../Benchmarks.md) for the reported runs. diff --git a/Tests/FluidUseTests/Fixtures/gliclass-hub-config.json b/Tests/FluidUseTests/Fixtures/gliclass-hub-config.json new file mode 100644 index 0000000..7183b3b --- /dev/null +++ b/Tests/FluidUseTests/Fixtures/gliclass-hub-config.json @@ -0,0 +1,46 @@ +{ + "format": "coreml", + "library_name": "coreml", + "pipeline": "dynamic-label-classification", + "source": { + "repo": "knowledgator/gliclass-edge-v3.0", + "revision": "df03993a2ed98e5e4a0d2dd7efbbd105abe874cf", + "license": "apache-2.0" + }, + "variant": "FluidUse application-tuned v2", + "parameters": 32705154, + "max_options": 25, + "sequence_format": "[CLS] (<