diff --git a/CLAUDE.md b/CLAUDE.md index b70ca89..98880da 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,7 +59,9 @@ Config directory is resolved as: `OMNI_CONFIG_DIR` > `XDG_CONFIG_HOME/omni-cli` All output is JSON to stdout. Errors go to stderr as JSON. Use `--compact` for non-indented output (good for piping to `jq`). -Nothing is written to stdout on failure: HTTP ≥400 bodies, error messages, and subcommand suggestions all go to stderr, and the exit code is non-zero. A failed API call leaves exactly one JSON document on stderr — `{"error": , "status": , "body": }` — so `2>err.json` stays parseable; nothing else is printed alongside it. Runtime errors don't print the usage block (flag-parse errors still do). +Nothing is written to stdout on failure: HTTP ≥400 bodies, error messages, and subcommand suggestions all go to stderr, and the exit code is non-zero. The one exception is a multi-job query stream where some jobs succeeded: the results that did decode are rendered and the failed jobs are reported on stderr, still exiting non-zero. A failed API call leaves exactly one JSON document on stderr — `{"error": , "status": , "body": }` — so `2>err.json` stays parseable; nothing else is printed alongside it. Runtime errors don't print the usage block (flag-parse errors still do). + +`query run` and `query wait` stream NDJSON with rows as base64 Arrow. For `--format human` and `--chart` the CLI decodes that stream (`internal/result`) and renders from the model's field metadata in `summary.fields` — label, `is_dimension`, `data_type`, `format` — polling `query/wait` until every job finishes. Nothing is inferred from names or values. `--chart` (on API command groups only, with `--chart-value`, `--chart-rows`) draws the Omni app's bar table: every dimension a column, every measure a column of bars scaled to its own maximum; `--chart-value` narrows the bars to the measures it names, in that order (on a pivot, the measures spread across the pivot values). A query whose `model_job.pivots` names result columns is reshaped from the stream's long-form rows (`Set.Pivot`) for both the table and the chart — the other dimensions as rows, a column per pivot value and measure (pivot value over measure label), pivot values in the order the stream's row groups agree on, falling back to the pivot fields' own sort (descending if the query sorts them so), capped at `column_limit` — and a measure's bars share one scale across its pivot columns; a chart drops columns that don't fit the terminal, with a note (a table shows every column up to `column_limit`). It is accepted only on commands whose spec returns the stream (`query run`, `query wait`) and refused before any request otherwise; a `resultType` in the body is dropped (with a note in human mode) since those documents carry no metadata; an explicitly requested JSON format (`--format json`, `OMNI_OUTPUT_FORMAT`, or the profile's `defaultOutputFormat`) is an error, while the JSON a pipe auto-resolves to is not, so `--chart | less` draws. `--workbook` (same groups) sets `workbookUrl: true` on a body whose spec declares it — refused with `planOnly`, and refused rather than silently dropped when there is no JSON object to set it on — and surfaces the `X-Omni-Workbook-Url` header: a line under human-rendered output, or on stderr (plain for a passed-through CSV/XLSX, `{"workbookUrl": …}` in JSON mode) so stdout stays the payload. Human rendering strips control characters from API and warehouse text (cell values, labels, JSON strings and keys, error details, the workbook link) so a value can't drive the terminal; JSON output and passed-through payloads are untouched. Stream rendering is buffered and written once, and a failing `query/wait` poll reports through the same error envelope as any API call. A 2xx body that isn't JSON (e.g. `query run`'s `text/ndjson` stream, or CSV/XLSX when `query run`'s body sets `"resultType"`) is passed through to stdout unchanged and counts as success. The body is read in full before anything is written, so a truncated response never leaves a partial payload on stdout. diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 688c3d8..318f2ce 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -17,6 +17,12 @@ make test # Run tests make clean # Remove built binary ``` +To check table and chart rendering by eye, `scripts/chart-demo.sh` runs read-only queries (tables, charts, pivots, a workbook link) against a sales pipeline demo model: + +```bash +OMNI_PROFILE=my-profile MODEL_ID= PAUSE=1 scripts/chart-demo.sh +``` + ## Updating the OpenAPI Spec The CLI auto-generates commands from the embedded OpenAPI spec. The default sync fetches it from `exploreomni/omni@main` via `gh`: diff --git a/README.md b/README.md index d44b2e7..2b47cda 100644 --- a/README.md +++ b/README.md @@ -202,6 +202,80 @@ A failed API call leaves exactly one JSON document on stderr, so `omni ... 2>err `body` holds the API's own payload and is omitted when the response wasn't JSON. A **successful** response that isn't JSON — `query run` streams `text/ndjson`, and returns CSV or XLSX with a result type — is passed through to stdout unchanged. +### Query results + +`query run` and `query wait` stream results as NDJSON with the rows as Arrow. In JSON mode that stream passes through untouched. In human mode the CLI decodes it and renders what the model says about each field — its label, whether it's a dimension or a measure, and its number format — so a `NUMBER_0` measure reads `12,526` and a `percent` measure reads `39.92%`, in the query's column order. If the first response's wait window elapses, the CLI polls `query/wait` until every job has finished. + +``` +╭───────────────┬──────────┬────────────────────╮ +│ Country │ Sessions │ Engaged Sessions % │ +├───────────────┼──────────┼────────────────────┤ +│ United States │ 12,526 │ 39.92% │ +│ Ireland │ 838 │ 44.87% │ +╰───────────────┴──────────┴────────────────────╯ +``` + +### Charts + +`--chart` draws the same results as the Omni app's bar table: every dimension is a column, and every measure gets a column of bars scaled to its own maximum, as the model defines them. + +```bash +omni query run --body @revenue-by-category.json --chart --workbook +``` + +``` +Category Total Sale Price +Jeans 1,602,513.81 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ +Accessories 955,617.30 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ +Outerwear & Coats 842,064.07 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ +Fashion Hoodies & Sweatshir… 756,824.63 ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ +Open in Omni: https://myorg.omniapp.co/e/1:abc123/1 +``` + +Narrow it to some of the measures by field or label — `--chart-value engaged_sessions_percent,"Sessions"`, `events_ext.sessions` and `sessions` all work; on a pivot, only those measures spread across the pivot values, so more of them fit — and cap the row count with `--chart-rows`. `--workbook` also opens the query in an ephemeral workbook: the link prints under the output (or, in JSON mode, as `{"workbookUrl": …}` on stderr, since stdout stays the API's payload). + +A query with `pivots` renders pivoted, as a table and as a chart: the remaining dimensions stay as rows, each pivot value heads its own columns, and a measure's bars share one scale across all of them. A chart drops columns that don't fit the terminal, with a note; a table shows every pivot column up to the query's `column_limit`. + +``` +Stage Closed Lost Closed Won Negotiation +Region Total amount +AMER $13,966,500 ████████████ $3,903,000 ███▍ $167,500 ▏ +EMEA $8,482,500 ███████▎ $1,949,500 █▋ - +APAC $3,919,500 ███▍ $1,591,500 █▍ $177,000 ▏ +``` + +Values that cross zero get a zero axis rather than being scaled against the maximum: + +``` +Created At Month Mom Change +Feb 2024 2,793.82 │▇▇▇▇▇ +Mar 2024 10,528.18 │▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ +Apr 2024 -68.70 ▇│ +Nov 2024 29,857.87 │▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ +``` + +A chart is drawn from the query stream's field metadata. A `resultType` in the body would replace that stream with a document, so `--chart` drops it and says so. Flags that can't work are refused before any request is made; a `--chart-value` is matched against the result's columns once it arrives. + +```console +$ omni query run --body @q.json --chart # body sets resultType +note: --chart ignores "resultType": "csv" and reads the query stream + +$ omni query run --body @q.json --chart --format json # or OMNI_OUTPUT_FORMAT=json +Error: --chart cannot be combined with JSON output: a chart is not JSON + +$ omni models list --chart +Error: --chart plots query results: use it with query run or query wait +``` + +Piping is fine — `omni ... --chart | less` still draws, since that JSON is auto-detected rather than asked for. Off a terminal — piped to a file, `pbcopy`, or a Slack message — the chart draws at 80 columns, which fits a code block. + +| Flag | Description | +|------|-------------| +| `--chart` | Draw query results as a bar table | +| `--chart-value FIELDS` | Only these measures, by field name or label; comma-separated or repeated (default: every measure) | +| `--chart-rows N` | Most rows to draw before summarising the rest (default 50) | +| `--workbook` | Also open the query in an ephemeral workbook and print its link | + ## Environment variables | Variable | Description | diff --git a/cmd/omni/agent_help.go b/cmd/omni/agent_help.go index e5e83d8..a97b144 100644 --- a/cmd/omni/agent_help.go +++ b/cmd/omni/agent_help.go @@ -151,6 +151,18 @@ available; binary values in its JSON object are interpreted as file paths. ## Common Flags --compact Non-indented JSON output + --chart Draw query run / query wait results as a terminal bar table: + every dimension a column, every measure a column of bars on + its own scale; a query with pivots is drawn pivoted. Leave + resultType out of the body. Cannot be combined with an + explicitly requested JSON format. + --chart-value FIELDS + Only these measures get bars, by field name or label; + comma-separated or repeated. Works on pivots too. + --chart-rows N Most rows to draw (default 50) + --workbook Also open the query in an ephemeral workbook; the link prints + under human output, or as {"workbookUrl": ...} on stderr in + JSON mode --token TOKEN API token (overrides env/config) --base-url URL API base URL (overrides config) --profile NAME Config profile to use diff --git a/cmd/omni/main.go b/cmd/omni/main.go index 5de5596..0ded339 100644 --- a/cmd/omni/main.go +++ b/cmd/omni/main.go @@ -1,15 +1,21 @@ package main import ( + "bytes" "embed" + "encoding/json" "errors" "fmt" + "io" + "net/http" "os" "runtime/debug" + "strings" "github.com/exploreomni/omni-cli/internal/auth" "github.com/exploreomni/omni-cli/internal/config" "github.com/exploreomni/omni-cli/internal/openapi" + "github.com/exploreomni/omni-cli/internal/output" "github.com/exploreomni/omni-cli/internal/updatecheck" "github.com/exploreomni/omni-cli/internal/useragent" "github.com/spf13/cobra" @@ -86,6 +92,7 @@ func main() { } for _, cmd := range apiCmds { + addResultFlags(cmd) root.AddCommand(cmd) } @@ -120,6 +127,143 @@ func addGlobalFlags(root *cobra.Command) { root.PersistentFlags().StringP("format", "o", "", "output format: json, human, auto (default auto: human on TTY, json when piped)") } +// addResultFlags registers the presentation flags on an API command group +// (not the root, so `config init --chart` is an error). Names are reserved +// in openapi.globalFlagKeys. +func addResultFlags(cmd *cobra.Command) { + f := cmd.PersistentFlags() + f.Bool("workbook", false, "also open the query in an ephemeral workbook and print its link") + f.Bool("chart", false, "draw query results as a bar chart") + f.StringSlice("chart-value", nil, "only these `fields` get bars, by field or label; comma-separated or repeated (default: every measure)") + f.Int("chart-rows", output.DefaultChartRows, "most rows to draw before summarising the rest") +} + +// chartOptions reads the --chart flags; nil when no chart was asked for. An +// explicitly chosen JSON format (flag, env, config — not a pipe's auto +// detection) refuses a chart. +func chartOptions(cmd *cobra.Command, chosenFormat string) (*output.ChartOptions, error) { + if on, err := cmd.Flags().GetBool("chart"); err != nil || !on { + for _, name := range []string{"chart-value", "chart-rows"} { + if cmd.Flags().Changed(name) { + return nil, fmt.Errorf("--%s only applies with --chart", name) + } + } + return nil, nil + } + if chosenFormat == config.FormatJSON { + return nil, fmt.Errorf("--chart cannot be combined with JSON output: a chart is not JSON") + } + values, _ := cmd.Flags().GetStringSlice("chart-value") + if len(values) == 0 { + values = nil + } + rows, _ := cmd.Flags().GetInt("chart-rows") + return &output.ChartOptions{ + Values: values, + Width: terminalWidth(), + MaxRows: rows, + }, nil +} + +// prepareBody applies --chart (drops any resultType, with a note) and +// --workbook (sets workbookUrl) to a JSON body, touching only fields the +// command's spec declares. +func prepareBody(chart, workbook bool, format string, cmd *cobra.Command, body []byte) ([]byte, error) { + if !chart && !workbook { + return body, nil + } + if workbook && !openapi.BodyDeclares(cmd, "workbookUrl") { + return nil, fmt.Errorf("--workbook is not supported by %s", cmd.CommandPath()) + } + if len(bytes.TrimSpace(body)) == 0 { + if workbook { + return nil, fmt.Errorf("--workbook needs a JSON request body to set workbookUrl on; pass one with --body or on stdin") + } + return body, nil + } + var obj map[string]json.RawMessage + if err := json.Unmarshal(body, &obj); err != nil { + // Silently dropping the flag would send the request without + // workbookUrl and leave the user wondering where their link went. + if workbook { + return nil, fmt.Errorf("--workbook needs a JSON object as the request body: %w", err) + } + return body, nil + } + if isTrue(obj["planOnly"]) { + if chart { + return nil, fmt.Errorf("--chart cannot be combined with planOnly") + } + return nil, fmt.Errorf("--workbook cannot be combined with planOnly") + } + changed := false + if raw, ok := obj["resultType"]; chart && ok && openapi.BodyDeclares(cmd, "resultType") { + if format == config.FormatHuman { + fmt.Fprintf(os.Stderr, "note: --chart ignores \"resultType\": %s and reads the query stream\n", raw) + } + delete(obj, "resultType") + changed = true + } + if !workbook { + if !changed { + return body, nil + } + filled, err := json.Marshal(obj) + if err != nil { + return nil, fmt.Errorf("preparing the request body: %w", err) + } + return filled, nil + } + if obj == nil { + return nil, fmt.Errorf("--workbook needs a JSON object as the request body") + } + obj["workbookUrl"] = json.RawMessage(`true`) + filled, err := json.Marshal(obj) + if err != nil { + return nil, fmt.Errorf("preparing the request body: %w", err) + } + return filled, nil +} + +// printWorkbookLink surfaces the X-Omni-Workbook-Url header. It joins +// human-rendered output on stdout; a passed-through body (CSV, XLSX) or +// JSON output keeps stdout as the payload, so the link goes to stderr. +func printWorkbookLink(resp *http.Response, format string, compact bool, stdout, stderr io.Writer) { + u := resp.Header.Get("X-Omni-Workbook-Url") + if u == "" { + return + } + rendered := strings.HasPrefix(resp.Header.Get("Content-Type"), "application/json") + switch { + case format == config.FormatHuman && rendered: + output.ChartLink(stdout, u) + case format == config.FormatHuman: + output.ChartLink(stderr, u) + default: + raw, _ := json.Marshal(map[string]string{"workbookUrl": u}) + _ = output.JSONBytes(stderr, raw, compact) + } +} + +func isTrue(raw json.RawMessage) bool { + var b bool + return json.Unmarshal(raw, &b) == nil && b +} + +func terminalWidth() int { + w, _, err := term.GetSize(int(os.Stdout.Fd())) + if err != nil || w <= 0 { + return 80 + } + if w < 40 { + return 40 + } + if w > 160 { + return 160 + } + return w +} + // executeAPICall is the callback invoked by generated commands to make the actual HTTP request. func executeAPICall(req openapi.APIRequest) error { cfg, err := resolveConfig(req.Cmd) @@ -129,7 +273,21 @@ func executeAPICall(req openapi.APIRequest) error { compact, _ := req.Cmd.Flags().GetBool("compact") formatFlag, _ := req.Cmd.Flags().GetString("format") - format := config.ResolveOutputFormat(formatFlag, term.IsTerminal(int(os.Stdout.Fd()))) + chosen := config.ChosenOutputFormat(formatFlag) + format := config.FormatFromChoice(chosen, term.IsTerminal(int(os.Stdout.Fd()))) + + chart, err := chartOptions(req.Cmd, chosen) + if err != nil { + return err + } + if chart != nil && !openapi.ReturnsStream(req.Cmd) { + return fmt.Errorf("--chart plots query results: use it with query run or query wait") + } + workbook, _ := req.Cmd.Flags().GetBool("workbook") + req.Body, err = prepareBody(chart != nil, workbook, format, req.Cmd, req.Body) + if err != nil { + return err + } // Show a spinner on stderr while the request is in flight. Only when the // user is at an interactive terminal AND they're going to see human output; @@ -143,7 +301,17 @@ func executeAPICall(req openapi.APIRequest) error { } defer resp.Body.Close() - err = outputResponse(resp, format, compact) + // A query stream rendered for a person is decoded — labels, formats, + // dimensions from the model — and waited on to completion. For JSON it + // passes through untouched, as the output contract promises. + if isQueryStream(resp) && (chart != nil || format == config.FormatHuman) { + err = renderStream(cfg, resp, format, compact, chart, os.Stdout, os.Stderr) + } else { + err = outputResponse(resp, format, compact, chart) + if err == nil { + printWorkbookLink(resp, format, compact, os.Stdout, os.Stderr) + } + } var apiErr *apiError if errors.As(err, &apiErr) { // outputResponse already wrote a complete error message to stderr — diff --git a/cmd/omni/output.go b/cmd/omni/output.go index 536c398..917df68 100644 --- a/cmd/omni/output.go +++ b/cmd/omni/output.go @@ -27,8 +27,8 @@ func (e *apiError) Error() string { return fmt.Sprintf("API returned HTTP %d", e.status) } -func outputResponse(resp *http.Response, format string, compact bool) error { - return outputResponseTo(os.Stdout, os.Stderr, resp, format, compact) +func outputResponse(resp *http.Response, format string, compact bool, chart *output.ChartOptions) error { + return outputResponseTo(os.Stdout, os.Stderr, resp, format, compact, chart) } // outputResponseTo writes a response to explicit streams, upholding two @@ -37,7 +37,7 @@ func outputResponse(resp *http.Response, format string, compact bool) error { // not an error, so it passes through to stdout byte for byte. The body is read // in full before anything is written, so a truncated read can't leave half a // payload on stdout ahead of a non-zero exit. -func outputResponseTo(stdout, stderr io.Writer, resp *http.Response, format string, compact bool) error { +func outputResponseTo(stdout, stderr io.Writer, resp *http.Response, format string, compact bool, chart *output.ChartOptions) error { data, err := io.ReadAll(resp.Body) if err != nil { // Still an envelope: JSON-mode stderr has to stay parseable even when @@ -64,6 +64,12 @@ func outputResponseTo(stdout, stderr io.Writer, resp *http.Response, format stri return nil } + // A chart asked of anything but a query stream is refused before the + // passthrough below, so a CSV isn't written out with --chart ignored. + if chart != nil { + return fmt.Errorf("--chart plots query results: this response is not a query stream") + } + // Non-JSON payloads (`query run`'s text/ndjson stream, or CSV/XLSX when // its body sets resultType) go out unchanged: no re-indenting, no appended newline, // so a redirect to a file reproduces the response byte for byte. diff --git a/cmd/omni/output_test.go b/cmd/omni/output_test.go index c1e96c8..8d1153f 100644 --- a/cmd/omni/output_test.go +++ b/cmd/omni/output_test.go @@ -7,8 +7,13 @@ import ( "fmt" "io" "net/http" + "reflect" "strings" "testing" + + "github.com/exploreomni/omni-cli/internal/openapi" + "github.com/exploreomni/omni-cli/internal/output" + "github.com/spf13/cobra" ) // These test the outputResponse function which is the last step before the @@ -22,7 +27,7 @@ func TestOutputResponse_Error(t *testing.T) { StatusCode: 400, Body: io.NopCloser(strings.NewReader(`{"error":"bad request"}`)), } - err := outputResponse(resp, "json", true) + err := outputResponse(resp, "json", true, nil) if err == nil { t.Fatal("expected error for 400 status") } @@ -37,7 +42,7 @@ func TestOutputResponse_NoContent(t *testing.T) { StatusCode: 204, Body: io.NopCloser(strings.NewReader("")), } - err := outputResponse(resp, "json", false) + err := outputResponse(resp, "json", false, nil) if err != nil { t.Fatalf("unexpected error: %v", err) } @@ -49,7 +54,7 @@ func TestOutputResponse_Error_Human(t *testing.T) { StatusCode: 404, Body: io.NopCloser(strings.NewReader(`{"detail":"not found"}`)), } - err := outputResponse(resp, "human", false) + err := outputResponse(resp, "human", false, nil) if err == nil { t.Fatal("expected error for 404 status") } @@ -66,7 +71,7 @@ func TestOutputResponseTo_ErrorBodyGoesToStderr(t *testing.T) { Body: io.NopCloser(strings.NewReader(`{"detail":"bad request"}`)), } - err := outputResponseTo(&stdout, &stderr, resp, "json", compact) + err := outputResponseTo(&stdout, &stderr, resp, "json", compact, nil) if err == nil { t.Fatalf("compact=%v: expected error for 400 status", compact) } @@ -87,7 +92,7 @@ func TestOutputResponseTo_HumanErrorGoesToStderr(t *testing.T) { Body: io.NopCloser(strings.NewReader(`{"detail":"not found"}`)), } - if err := outputResponseTo(&stdout, &stderr, resp, "human", false); err == nil { + if err := outputResponseTo(&stdout, &stderr, resp, "human", false, nil); err == nil { t.Fatal("expected error for 404 status") } if stdout.Len() != 0 { @@ -109,7 +114,7 @@ func TestOutputResponseTo_StderrIsSingleJSONDocument(t *testing.T) { Body: io.NopCloser(strings.NewReader(`{"detail":"bad model id","code":"INVALID"}`)), } - err := outputResponseTo(&stdout, &stderr, resp, "json", compact) + err := outputResponseTo(&stdout, &stderr, resp, "json", compact, nil) // The caller silences cobra's duplicate line off the back of this type. var apiErr *apiError @@ -149,7 +154,7 @@ func TestOutputResponseTo_NonJSONErrorBodyStillJSON(t *testing.T) { Body: io.NopCloser(strings.NewReader("Bad Gateway")), } - if err := outputResponseTo(&stdout, &stderr, resp, "json", true); err == nil { + if err := outputResponseTo(&stdout, &stderr, resp, "json", true, nil); err == nil { t.Fatal("expected error for 502 status") } var envelope struct { @@ -196,7 +201,7 @@ func TestOutputResponseTo_NonJSONSuccessPassesThrough(t *testing.T) { Body: io.NopCloser(strings.NewReader(tc.body)), } - if err := outputResponseTo(&stdout, &stderr, resp, "json", compact); err != nil { + if err := outputResponseTo(&stdout, &stderr, resp, "json", compact, nil); err != nil { t.Fatalf("%s compact=%v: non-JSON 2xx body should succeed, got %v", tc.name, compact, err) } if stdout.String() != tc.body { @@ -219,7 +224,7 @@ func TestOutputResponseTo_NonJSONSuccessPassesThroughHuman(t *testing.T) { Body: io.NopCloser(strings.NewReader(body)), } - if err := outputResponseTo(&stdout, &stderr, resp, "human", false); err != nil { + if err := outputResponseTo(&stdout, &stderr, resp, "human", false, nil); err != nil { t.Fatalf("non-JSON 2xx body should succeed, got %v", err) } if stdout.String() != body { @@ -237,7 +242,7 @@ func TestOutputResponseTo_NullErrorBodyOmitted(t *testing.T) { Body: io.NopCloser(strings.NewReader("null")), } - if err := outputResponseTo(&stdout, &stderr, resp, "json", compact); err == nil { + if err := outputResponseTo(&stdout, &stderr, resp, "json", compact, nil); err == nil { t.Fatalf("compact=%v: expected error for 500 status", compact) } var envelope map[string]any @@ -262,7 +267,7 @@ func TestOutputResponseTo_ReadFailureWritesNothing(t *testing.T) { Body: io.NopCloser(&truncatedReader{data: []byte(`{"records":[`)}), } - err := outputResponseTo(&stdout, &stderr, resp, "json", false) + err := outputResponseTo(&stdout, &stderr, resp, "json", false, nil) var apiErr *apiError if !errors.As(err, &apiErr) { t.Fatalf("error = %v, want *apiError so cobra's duplicate line is silenced", err) @@ -322,7 +327,7 @@ func TestOutputResponseTo_SuccessGoesToStdout(t *testing.T) { Body: io.NopCloser(strings.NewReader(tc.body)), } - if err := outputResponseTo(&stdout, &stderr, resp, tc.format, true); err != nil { + if err := outputResponseTo(&stdout, &stderr, resp, tc.format, true, nil); err != nil { t.Fatalf("unexpected error: %v", err) } if !strings.Contains(stdout.String(), tc.want) { @@ -346,3 +351,200 @@ func TestExtractErrorDetail_NestedErrorObject(t *testing.T) { t.Errorf("extractErrorDetail = %q, want raw body fallback", got) } } + +func TestChartOptions(t *testing.T) { + tests := []struct { + name string + args []string + format string + want *output.ChartOptions + errs bool + }{ + {name: "absent", args: nil}, + {name: "flag", args: []string{"--chart"}, want: &output.ChartOptions{}}, + { + name: "columns", + args: []string{"--chart", "--chart-value", "revenue,count", "--chart-value", "Win rate"}, + want: &output.ChartOptions{Values: []string{"revenue", "count", "Win rate"}}, + }, + {name: "rejected with json", args: []string{"--chart"}, format: "json", errs: true}, + {name: "allowed with human", args: []string{"--chart"}, format: "human", want: &output.ChartOptions{}}, + {name: "chart-value without chart", args: []string{"--chart-value", "revenue"}, errs: true}, + {name: "chart-rows without chart", args: []string{"--chart-rows", "5"}, errs: true}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + group := &cobra.Command{Use: "query"} + addResultFlags(group) + if err := group.ParseFlags(tc.args); err != nil { + t.Fatalf("parsing %v: %v", tc.args, err) + } + got, err := chartOptions(group, tc.format) + if tc.errs { + if err == nil { + t.Fatal("expected an error") + } + return + } + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if tc.want == nil { + if got != nil { + t.Fatalf("expected no chart, got %+v", got) + } + return + } + if got == nil { + t.Fatal("expected chart options, got nil") + } + // Width comes from the terminal; the rest is the flags. + got.Width = 0 + got.MaxRows = 0 + if !reflect.DeepEqual(got, tc.want) { + t.Errorf("got %+v, want %+v", *got, *tc.want) + } + }) + } +} + +func TestPrepareBody(t *testing.T) { + queryRun := map[string]string{openapi.BodyPropsAnnotation: "query,resultType,workbookUrl,planOnly"} + generate := map[string]string{openapi.BodyPropsAnnotation: "modelId,prompt"} + tests := []struct { + name string + chart, workbook bool + props map[string]string + body string + want string + err string + }{ + {name: "nothing asked", props: queryRun, body: `{"query":{}}`, want: `{"query":{}}`}, + {name: "chart leaves the body alone", chart: true, props: queryRun, body: `{"query":{"limit":5}}`, want: `{"query":{"limit":5}}`}, + {name: "chart drops a json resultType", chart: true, props: queryRun, body: `{"query":{},"resultType":"json"}`, want: `{"query":{}}`}, + {name: "chart drops a csv resultType", chart: true, props: queryRun, body: `{"query":{},"resultType":"csv"}`, want: `{"query":{}}`}, + {name: "workbook sets the field", workbook: true, props: queryRun, body: `{"query":{}}`, want: `{"query":{},"workbookUrl":true}`}, + {name: "workbook flag wins over a false in the body", workbook: true, props: queryRun, body: `{"query":{},"workbookUrl":false}`, want: `{"query":{},"workbookUrl":true}`}, + {name: "workbook with a resultType is fine", workbook: true, props: queryRun, body: `{"query":{},"resultType":"csv"}`, want: `{"query":{},"resultType":"csv","workbookUrl":true}`}, + {name: "workbook and planOnly conflict", workbook: true, props: queryRun, body: `{"query":{},"planOnly":true}`, err: "planOnly"}, + {name: "chart and planOnly conflict", chart: true, props: queryRun, body: `{"query":{},"planOnly":true}`, err: "--chart cannot be combined with planOnly"}, + {name: "chart with planOnly false", chart: true, props: queryRun, body: `{"query":{},"planOnly":false}`, want: `{"query":{},"planOnly":false}`}, + {name: "chart keeps an undeclared resultType", chart: true, props: generate, body: `{"modelId":"x","resultType":"csv"}`, want: `{"modelId":"x","resultType":"csv"}`}, + {name: "workbook on a command without the field", workbook: true, props: generate, body: `{"modelId":"x"}`, err: "not supported"}, + {name: "chart on a command without resultType", chart: true, props: generate, body: `{"modelId":"x"}`, want: `{"modelId":"x"}`}, + {name: "no body", chart: true, props: queryRun, want: ``}, + {name: "not JSON", chart: true, props: queryRun, body: `not json`, want: `not json`}, + // --workbook has nowhere to put workbookUrl: say so rather than + // sending the request and losing the link silently. + {name: "workbook with no body", workbook: true, props: queryRun, err: "needs a JSON request body"}, + {name: "workbook with a non-JSON body", workbook: true, props: queryRun, body: `not json`, err: "needs a JSON object"}, + {name: "workbook with null", workbook: true, props: queryRun, body: `null`, err: "needs a JSON object"}, + {name: "chart and workbook with null", chart: true, workbook: true, props: queryRun, body: `null`, err: "needs a JSON object"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + cmd := &cobra.Command{Use: "run", Annotations: tc.props} + got, err := prepareBody(tc.chart, tc.workbook, "human", cmd, []byte(tc.body)) + if tc.err != "" { + if err == nil { + t.Fatalf("expected an error, got body %s", got) + } + if !strings.Contains(err.Error(), tc.err) { + t.Errorf("error %q does not mention %q", err, tc.err) + } + return + } + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if !sameJSON(t, got, []byte(tc.want)) { + t.Errorf("got %s, want %s", got, tc.want) + } + }) + } +} + +// The workbook link rides on a header: under the output for a person, on +// stderr as JSON for a machine, nowhere when the API didn't send one. +func TestPrintWorkbookLink(t *testing.T) { + mk := func(u, ct string) *http.Response { + h := http.Header{"Content-Type": []string{ct}} + if u != "" { + h.Set("X-Omni-Workbook-Url", u) + } + return &http.Response{Header: h} + } + var stdout, stderr bytes.Buffer + printWorkbookLink(mk("https://x/e/1", "application/json"), "human", false, &stdout, &stderr) + if !strings.Contains(stdout.String(), "Open in Omni: https://x/e/1") || stderr.Len() != 0 { + t.Errorf("human: stdout=%q stderr=%q", stdout.String(), stderr.String()) + } + // A passed-through download keeps stdout as the file. + stdout.Reset() + stderr.Reset() + printWorkbookLink(mk("https://x/e/1", "text/csv"), "human", false, &stdout, &stderr) + if stdout.Len() != 0 || !strings.Contains(stderr.String(), "Open in Omni: https://x/e/1") { + t.Errorf("csv: stdout=%q stderr=%q", stdout.String(), stderr.String()) + } + stdout.Reset() + stderr.Reset() + printWorkbookLink(mk("https://x/e/1", "application/json"), "json", true, &stdout, &stderr) + if stdout.Len() != 0 || strings.TrimSpace(stderr.String()) != `{"workbookUrl":"https://x/e/1"}` { + t.Errorf("json: stdout=%q stderr=%q", stdout.String(), stderr.String()) + } + stdout.Reset() + stderr.Reset() + printWorkbookLink(mk("", "application/json"), "human", false, &stdout, &stderr) + if stdout.Len() != 0 || stderr.Len() != 0 { + t.Errorf("no header: stdout=%q stderr=%q", stdout.String(), stderr.String()) + } +} + +// --chart on anything that isn't a query stream is refused: there is no +// field metadata to draw from. +func TestOutputResponse_ChartNeedsAStream(t *testing.T) { + var stdout, stderr bytes.Buffer + resp := &http.Response{ + StatusCode: 200, + Header: http.Header{"Content-Type": []string{"application/json"}}, + Body: io.NopCloser(strings.NewReader(`[{"region":"east","revenue":10}]`)), + } + err := outputResponseTo(&stdout, &stderr, resp, "human", false, &output.ChartOptions{Width: 60}) + if err == nil || !strings.Contains(err.Error(), "not a query stream") { + t.Fatalf("expected a refusal, got %v", err) + } + if stdout.Len() != 0 { + t.Errorf("nothing should reach stdout, got %q", stdout.String()) + } +} + +// The same refusal holds for a non-JSON 2xx body: a CSV isn't a stream +// either, so it is refused rather than written out with --chart ignored. +func TestOutputResponse_ChartRefusedBeforePassthrough(t *testing.T) { + var stdout, stderr bytes.Buffer + resp := &http.Response{ + StatusCode: 200, + Header: http.Header{"Content-Type": []string{"text/csv"}}, + Body: io.NopCloser(strings.NewReader("region,revenue\neast,10\n")), + } + err := outputResponseTo(&stdout, &stderr, resp, "human", false, &output.ChartOptions{Width: 60}) + if err == nil || !strings.Contains(err.Error(), "not a query stream") { + t.Fatalf("expected a refusal, got %v", err) + } + if stdout.Len() != 0 { + t.Errorf("nothing should reach stdout, got %q", stdout.String()) + } +} + +// sameJSON compares two bodies structurally, falling back to bytes when +// either isn't JSON. +func sameJSON(t *testing.T, a, b []byte) bool { + t.Helper() + var x, y any + if json.Unmarshal(a, &x) != nil || json.Unmarshal(b, &y) != nil { + return bytes.Equal(bytes.TrimSpace(a), bytes.TrimSpace(b)) + } + ja, _ := json.Marshal(x) + jb, _ := json.Marshal(y) + return bytes.Equal(ja, jb) +} diff --git a/cmd/omni/spinner.go b/cmd/omni/spinner.go index ce21c5c..c69b90b 100644 --- a/cmd/omni/spinner.go +++ b/cmd/omni/spinner.go @@ -1,6 +1,7 @@ package main import ( + "fmt" "math/rand" "os" "time" @@ -43,6 +44,8 @@ func (h *spinnerHandle) Stop() { } close(h.stop) h.s.Stop() + // Clear the whole line: the library only erases its last frame. + fmt.Fprint(os.Stderr, "\r\033[2K") } // maybeStartSpinner returns a running spinner, or nil if the environment diff --git a/cmd/omni/stream.go b/cmd/omni/stream.go new file mode 100644 index 0000000..b175b99 --- /dev/null +++ b/cmd/omni/stream.go @@ -0,0 +1,113 @@ +package main + +import ( + "bytes" + "fmt" + "io" + "net/http" + "net/url" + "strings" + + "github.com/exploreomni/omni-cli/internal/auth" + "github.com/exploreomni/omni-cli/internal/config" + "github.com/exploreomni/omni-cli/internal/output" + "github.com/exploreomni/omni-cli/internal/result" +) + +// maxWaitPolls bounds re-waits; each query/wait call blocks server-side. +const maxWaitPolls = 30 + +func isQueryStream(resp *http.Response) bool { + return resp.StatusCode < 400 && strings.HasPrefix(resp.Header.Get("Content-Type"), "text/ndjson") +} + +// renderStream renders a query stream as a table or chart, waiting on any +// job still running. JSON output never comes here. +func renderStream(cfg *config.ResolvedConfig, resp *http.Response, format string, compact bool, chart *output.ChartOptions, stdout, stderr io.Writer) error { + data, err := io.ReadAll(resp.Body) + if err != nil { + return fmt.Errorf("reading response: %w", err) + } + st, err := result.Parse(data) + if err != nil { + return err + } + + for polls := 0; len(st.Remaining) > 0; polls++ { + if polls >= maxWaitPolls { + return fmt.Errorf("query still running after %d waits (jobs %s); try again with `omni query wait --job-ids %s`", + polls, strings.Join(st.Remaining, ","), strings.Join(st.Remaining, ",")) + } + more, err := waitForJobs(cfg, format, compact, stderr, st.Remaining) + if err != nil { + return err + } + st.Sets = append(st.Sets, more.Sets...) + st.Failures = append(st.Failures, more.Failures...) + st.Remaining = more.Remaining + } + + // A job that failed doesn't void the ones that didn't: render what + // completed, then report the failure and exit non-zero. With nothing + // rendered, stdout stays empty and only the failure is reported. + failed := st.Err() + if len(st.Sets) == 0 && failed != nil { + return reportFailures(stderr, format, compact, failed) + } + + // Render everything before writing anything, so a failure on a later + // set leaves stdout empty rather than half a result. + var out bytes.Buffer + if len(st.Sets) == 0 { + fmt.Fprintln(&out, "No results.") + } + for i, set := range st.Sets { + if i > 0 { + fmt.Fprintln(&out) + } + if chart != nil { + if err := output.ResultChart(&out, set, *chart); err != nil { + return err + } + } else { + output.ResultTable(&out, set) + } + } + if u := resp.Header.Get("X-Omni-Workbook-Url"); u != "" { + output.ChartLink(&out, u) + } + if _, err := stdout.Write(out.Bytes()); err != nil { + return err + } + if failed != nil { + return reportFailures(stderr, format, compact, failed) + } + return nil +} + +// reportFailures reports failed jobs like a failed API call, with no HTTP status. +func reportFailures(stderr io.Writer, format string, compact bool, failed error) error { + writeError(stderr, format, 0, failed.Error(), nil, compact) + return &apiError{detail: failed.Error()} +} + +func waitForJobs(cfg *config.ResolvedConfig, format string, compact bool, stderr io.Writer, ids []string) (*result.Stream, error) { + sp := maybeStartSpinner(format) + resp, err := auth.Do(cfg, http.MethodGet, "/api/v1/query/wait?jobIds="+url.QueryEscape(strings.Join(ids, ",")), nil) + sp.Stop() + if err != nil { + return nil, err + } + defer resp.Body.Close() + data, err := io.ReadAll(resp.Body) + if err != nil { + return nil, fmt.Errorf("reading query/wait response: %w", err) + } + if resp.StatusCode >= 400 { + body := jsonBody(data) + detail := extractErrorDetail(body, data, resp.StatusCode) + writeError(stderr, format, resp.StatusCode, detail, body, compact) + return nil, &apiError{status: resp.StatusCode} + } + return result.Parse(data) +} diff --git a/cmd/omni/stream_test.go b/cmd/omni/stream_test.go new file mode 100644 index 0000000..697e9fb --- /dev/null +++ b/cmd/omni/stream_test.go @@ -0,0 +1,223 @@ +package main + +import ( + "bytes" + "encoding/base64" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/apache/arrow-go/v18/arrow" + "github.com/apache/arrow-go/v18/arrow/array" + "github.com/apache/arrow-go/v18/arrow/ipc" + "github.com/apache/arrow-go/v18/arrow/memory" + "github.com/exploreomni/omni-cli/internal/config" + "github.com/exploreomni/omni-cli/internal/output" +) + +func completedJob(t *testing.T, id string) string { + t.Helper() + mem := memory.NewGoAllocator() + schema := arrow.NewSchema([]arrow.Field{ + {Name: "e.country", Type: arrow.BinaryTypes.String, Nullable: true}, + {Name: "e.sessions", Type: arrow.PrimitiveTypes.Int64, Nullable: true}, + }, nil) + b := array.NewRecordBuilder(mem, schema) + defer b.Release() + b.Field(0).(*array.StringBuilder).AppendValues([]string{"United States", "Ireland"}, nil) + b.Field(1).(*array.Int64Builder).AppendValues([]int64{12526, 838}, nil) + rec := b.NewRecordBatch() + defer rec.Release() + var buf bytes.Buffer + w := ipc.NewWriter(&buf, ipc.WithSchema(schema)) + if err := w.Write(rec); err != nil { + t.Fatal(err) + } + w.Close() + + line, _ := json.Marshal(map[string]any{ + "job_id": id, "status": "COMPLETE", + "summary": map[string]any{"fields": map[string]any{ + "e.country": map[string]any{"label": "Country", "is_dimension": true, "data_type": "STRING"}, + "e.sessions": map[string]any{"label": "Sessions", "data_type": "NUMBER", "format": map[string]any{"value": "NUMBER_0"}}, + }}, + "result": base64.StdEncoding.EncodeToString(buf.Bytes()), + }) + return string(line) +} + +func streamResp(body string, hdr map[string]string) *http.Response { + h := http.Header{"Content-Type": []string{"text/ndjson"}} + for k, v := range hdr { + h.Set(k, v) + } + return &http.Response{StatusCode: 200, Header: h, Body: io.NopCloser(strings.NewReader(body))} +} + +// A stream whose wait window elapsed is followed up on query/wait until +// every job has finished, and the result renders as if it had come at once. +func TestRenderStream_PollsUntilComplete(t *testing.T) { + var waits []string + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/api/v1/query/wait" { + t.Errorf("unexpected request %s", r.URL) + } + waits = append(waits, r.URL.Query().Get("jobIds")) + w.Header().Set("Content-Type", "text/ndjson") + if len(waits) == 1 { + fmt.Fprintln(w, `{"remaining_job_ids":["j1"],"timed_out":"true"}`) + return + } + fmt.Fprintln(w, completedJob(t, "j1")) + fmt.Fprintln(w, `{"remaining_job_ids":[],"timed_out":"false"}`) + })) + defer srv.Close() + cfg := &config.ResolvedConfig{BaseURL: srv.URL, Token: "t"} + + first := `{"jobs_submitted":{"j1":"r1"}}` + "\n" + `{"remaining_job_ids":["j1"],"timed_out":"true"}` + "\n" + var stdout bytes.Buffer + var stderr bytes.Buffer + err := renderStream(cfg, streamResp(first, map[string]string{"X-Omni-Workbook-Url": "https://acme.omniapp.co/e/1:abc/1"}), "human", false, &output.ChartOptions{Width: 60}, &stdout, &stderr) + if err != nil { + t.Fatalf("renderStream: %v", err) + } + if len(waits) != 2 || waits[0] != "j1" { + t.Errorf("expected two waits on j1, got %v", waits) + } + out := stdout.String() + for _, want := range []string{"United States", "12,526", "▇", "Open in Omni: https://acme.omniapp.co/e/1:abc/1"} { + if !strings.Contains(out, want) { + t.Errorf("output missing %q:\n%s", want, out) + } + } +} + +// Without --chart, a stream in human mode is the model's table. +func TestRenderStream_TableWhenNotCharting(t *testing.T) { + body := `{"jobs_submitted":{"j1":"r1"}}` + "\n" + completedJob(t, "j1") + "\n" + `{"remaining_job_ids":[],"timed_out":"false"}` + "\n" + var stdout bytes.Buffer + if err := renderStream(&config.ResolvedConfig{}, streamResp(body, nil), "human", false, nil, &stdout, io.Discard); err != nil { + t.Fatal(err) + } + out := stdout.String() + if !strings.Contains(out, "Country") || !strings.Contains(out, "12,526") || strings.Contains(out, "▇") { + t.Errorf("expected a table, got:\n%s", out) + } + if strings.Contains(out, "Open in Omni") { + t.Errorf("no link header, no link line:\n%s", out) + } +} + +func TestRenderStream_FailedJob(t *testing.T) { + body := `{"job_id":"j1","status":"ERROR","error":{"message":"No such field \"e.nope\""}}` + "\n" + var stdout bytes.Buffer + err := renderStream(&config.ResolvedConfig{}, streamResp(body, nil), "human", false, nil, &stdout, io.Discard) + if err == nil || !strings.Contains(err.Error(), `No such field "e.nope"`) { + t.Fatalf("expected the API's message, got %v", err) + } + if stdout.Len() != 0 { + t.Errorf("nothing should reach stdout, got %q", stdout.String()) + } +} + +// A failing query/wait poll reports like any failed API call: one JSON +// envelope on stderr in JSON mode, nothing on stdout. +func TestRenderStream_WaitErrorUsesTheEnvelope(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(401) + fmt.Fprint(w, `{"error":{"code":401,"message":"token expired"}}`) + })) + defer srv.Close() + first := `{"jobs_submitted":{"j1":"r1"}}` + "\n" + `{"remaining_job_ids":["j1"],"timed_out":"true"}` + "\n" + var stdout, stderr bytes.Buffer + err := renderStream(&config.ResolvedConfig{BaseURL: srv.URL, Token: "t"}, streamResp(first, nil), "json", true, &output.ChartOptions{Width: 60}, &stdout, &stderr) + var apiErr *apiError + if !errors.As(err, &apiErr) || apiErr.status != 401 { + t.Fatalf("expected an apiError 401, got %v", err) + } + if stdout.Len() != 0 { + t.Errorf("nothing should reach stdout, got %q", stdout.String()) + } + var env map[string]any + if json.Unmarshal(stderr.Bytes(), &env) != nil || env["status"] != float64(401) || env["error"] != "token expired" { + t.Errorf("stderr should be one JSON envelope, got %q", stderr.String()) + } +} + +// One failed job is reported and exits non-zero, but the jobs that did +// complete are still rendered rather than thrown away with it. +func TestRenderStream_FailedJobKeepsTheOthers(t *testing.T) { + body := `{"jobs_submitted":{"j1":"r1","j2":"r2"}}` + "\n" + completedJob(t, "j1") + "\n" + + `{"job_id":"j2","status":"ERROR","error_type":"PLAN","error_message":"No such view"}` + "\n" + var stdout bytes.Buffer + err := renderStream(&config.ResolvedConfig{}, streamResp(body, nil), "human", false, &output.ChartOptions{Width: 60}, &stdout, io.Discard) + if err == nil || !strings.Contains(err.Error(), "No such view") { + t.Fatalf("expected the second job's error, got %v", err) + } + if !strings.Contains(stdout.String(), "Ireland") { + t.Errorf("the completed job should still render, got %q", stdout.String()) + } +} + +func TestRenderStream_FailedJobUsesTheEnvelope(t *testing.T) { + failed := `{"job_id":"j2","status":"ERROR","error_type":"PLAN","error_message":"No such view"}` + for name, body := range map[string]string{ + "only failures": `{"jobs_submitted":{"j2":"r2"}}` + "\n" + failed + "\n", + "mixed": `{"jobs_submitted":{"j1":"r1","j2":"r2"}}` + "\n" + completedJob(t, "j1") + "\n" + failed + "\n", + } { + t.Run(name, func(t *testing.T) { + var stdout, stderr bytes.Buffer + err := renderStream(&config.ResolvedConfig{}, streamResp(body, nil), "json", true, &output.ChartOptions{Width: 60}, &stdout, &stderr) + var apiErr *apiError + if !errors.As(err, &apiErr) { + t.Fatalf("expected an apiError, got %v", err) + } + var env map[string]any + if json.Unmarshal(stderr.Bytes(), &env) != nil || !strings.Contains(fmt.Sprint(env["error"]), "No such view") { + t.Errorf("stderr should be one JSON envelope, got %q", stderr.String()) + } + }) + } +} + +func TestRenderStream_FailedJobIsSanitized(t *testing.T) { + body := `{"jobs_submitted":{"j1":"r1"}}` + "\n" + + `{"job_id":"j1","status":"ERROR","error_message":"bad\u001b]0;pwned\u0007 view"}` + "\n" + var stderr bytes.Buffer + _ = renderStream(&config.ResolvedConfig{}, streamResp(body, nil), "human", false, nil, io.Discard, &stderr) + if strings.ContainsAny(stderr.String(), "\x1b\x07") || !strings.Contains(stderr.String(), "view") { + t.Errorf("control characters should be stripped, got %q", stderr.String()) + } +} + +// With nothing left to render, stdout stays empty. +func TestRenderStream_OnlyFailuresWriteNothing(t *testing.T) { + body := `{"jobs_submitted":{"j1":"r1"}}` + "\n" + + `{"job_id":"j1","status":"ERROR","error_type":"PLAN","error_message":"No such view"}` + "\n" + var stdout bytes.Buffer + err := renderStream(&config.ResolvedConfig{}, streamResp(body, nil), "human", false, nil, &stdout, io.Discard) + if err == nil || !strings.Contains(err.Error(), "No such view") { + t.Fatalf("expected the job's error, got %v", err) + } + if stdout.Len() != 0 { + t.Errorf("nothing should reach stdout, got %q", stdout.String()) + } +} + +func TestIsQueryStream(t *testing.T) { + if !isQueryStream(streamResp("", nil)) { + t.Error("text/ndjson 200 is a stream") + } + if isQueryStream(&http.Response{StatusCode: 400, Header: http.Header{"Content-Type": []string{"text/ndjson"}}}) { + t.Error("a failure is not a stream to render") + } + if isQueryStream(&http.Response{StatusCode: 200, Header: http.Header{"Content-Type": []string{"application/json"}}}) { + t.Error("JSON is not a stream") + } +} diff --git a/go.mod b/go.mod index 2c6c3c6..6faa72b 100644 --- a/go.mod +++ b/go.mod @@ -3,15 +3,17 @@ module github.com/exploreomni/omni-cli go 1.25.6 require ( + github.com/apache/arrow-go/v18 v18.8.0 github.com/briandowns/spinner v1.23.2 github.com/charmbracelet/lipgloss v1.1.0 + github.com/charmbracelet/x/ansi v0.8.0 github.com/gofrs/flock v0.13.0 github.com/pb33f/libopenapi v0.34.4 github.com/spf13/cobra v1.10.2 github.com/spf13/pflag v1.0.9 go.yaml.in/yaml/v4 v4.0.0-rc.4 golang.org/x/oauth2 v0.36.0 - golang.org/x/term v0.41.0 + golang.org/x/term v0.45.0 ) require ( @@ -19,20 +21,27 @@ require ( github.com/bahlo/generic-list-go v0.2.0 // indirect github.com/buger/jsonparser v1.1.2 // indirect github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc // indirect - github.com/charmbracelet/x/ansi v0.8.0 // indirect github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd // indirect github.com/charmbracelet/x/term v0.2.1 // indirect + github.com/clipperhouse/uax29/v2 v2.7.0 // indirect github.com/fatih/color v1.7.0 // indirect + github.com/goccy/go-json v0.10.6 // indirect + github.com/google/flatbuffers v25.12.19+incompatible // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/klauspost/compress v1.19.2 // indirect + github.com/klauspost/cpuid/v2 v2.4.0 // indirect github.com/lucasb-eyer/go-colorful v1.2.0 // indirect github.com/mattn/go-colorable v0.1.2 // indirect - github.com/mattn/go-isatty v0.0.20 // indirect - github.com/mattn/go-runewidth v0.0.16 // indirect + github.com/mattn/go-isatty v0.0.24 // indirect + github.com/mattn/go-runewidth v0.0.20 // indirect github.com/muesli/termenv v0.16.0 // indirect github.com/pb33f/jsonpath v0.8.2 // indirect github.com/pb33f/ordered-map/v2 v2.3.1 // indirect + github.com/pierrec/lz4/v4 v4.1.29 // indirect github.com/rivo/uniseg v0.4.7 // indirect github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect - golang.org/x/sync v0.20.0 // indirect - golang.org/x/sys v0.42.0 // indirect + github.com/zeebo/xxh3 v1.1.0 // indirect + golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect + golang.org/x/sync v0.22.0 // indirect + golang.org/x/sys v0.47.0 // indirect ) diff --git a/go.sum b/go.sum index 461a9e6..33182f9 100644 --- a/go.sum +++ b/go.sum @@ -1,3 +1,9 @@ +github.com/andybalholm/brotli v1.2.3 h1:8H1qwOkl2LPfjf3YezB90JnCliZb6SInJ/OJkEbA5NQ= +github.com/andybalholm/brotli v1.2.3/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY= +github.com/apache/arrow-go/v18 v18.8.0 h1:BLOzbPv7bxMPgXPacAg6HQjnxupYsZzC4tf+FkqPU/M= +github.com/apache/arrow-go/v18 v18.8.0/go.mod h1:uJCFfCwq0KsxCmsCfQg4ft+LsW+iHYzAXiSDh5ug/8U= +github.com/apache/thrift v0.24.0 h1:zy31L1a49QTNB2bG1BBfMXol3yJrTH975G3pPubQVLQ= +github.com/apache/thrift v0.24.0/go.mod h1:zPt6WxgvTOM6hF92y8C+MkEM5LMxZuk4JcQOiU4Esvs= github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k= github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8= github.com/aymanbagabas/go-udiff v0.2.0 h1:TK0fH4MteXUDspT88n8CKzvK0X9O2xu9yQjWpi6yML8= @@ -20,24 +26,34 @@ github.com/charmbracelet/x/exp/golden v0.0.0-20240806155701-69247e0abc2a h1:G99k github.com/charmbracelet/x/exp/golden v0.0.0-20240806155701-69247e0abc2a/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U= github.com/charmbracelet/x/term v0.2.1 h1:AQeHeLZ1OqSXhrAWpYUtZyX1T3zVxfpZuEQMIQaGIAQ= github.com/charmbracelet/x/term v0.2.1/go.mod h1:oQ4enTYFV7QN4m0i9mzHrViD7TQKvNEEkHUMCmsxdUg= +github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk= +github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= -github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= -github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= +github.com/goccy/go-json v0.10.6 h1:p8HrPJzOakx/mn/bQtjgNjdTcN+/S6FcG2CTtQOrHVU= +github.com/goccy/go-json v0.10.6/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M= github.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw= github.com/gofrs/flock v0.13.0/go.mod h1:jxeyy9R1auM5S6JYDBhDt+E2TCo7DkratH4Pgi8P+Z0= +github.com/google/flatbuffers v25.12.19+incompatible h1:haMV2JRRJCe1998HeW/p0X9UaMTK6SDo0ffLn2+DbLs= +github.com/google/flatbuffers v25.12.19+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/klauspost/compress v1.19.2 h1:hMRETovs/pu/dVWN7zIT1PGG8t509MwT6bO7XSi26R8= +github.com/klauspost/compress v1.19.2/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= +github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw= +github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU= github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY= github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0= github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU= github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= -github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= -github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= -github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= -github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= +github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI= +github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A= +github.com/mattn/go-runewidth v0.0.20 h1:WcT52H91ZUAwy8+HUkdM3THM6gXqXuLJi9O3rjcQQaQ= +github.com/mattn/go-runewidth v0.0.20/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs= github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc= github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk= github.com/pb33f/jsonpath v0.8.2 h1:Ou4C7zjYClBm97dfZjDCjdZGusJoynv/vrtiEKNfj2Y= @@ -46,9 +62,8 @@ github.com/pb33f/libopenapi v0.34.4 h1:BWWXA3U4SlsHEvfczk+DJHu2O38ktgKw+zBEYaDZ2 github.com/pb33f/libopenapi v0.34.4/go.mod h1:MsDdUlQ1CdrIDO5v26JfgBxQs7kcaOUEpMP3EqU6bI4= github.com/pb33f/ordered-map/v2 v2.3.1 h1:5319HDO0aw4DA4gzi+zv4FXU9UlSs3xGZ40wcP1nBjY= github.com/pb33f/ordered-map/v2 v2.3.1/go.mod h1:qxFQgd0PkVUtOMCkTapqotNgzRhMPL7VvaHKbd1HnmQ= -github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= -github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= +github.com/pierrec/lz4/v4 v4.1.29 h1:CDQY6qZOLI4DW0Nx6R1vRrifrCeQHnNXkMb0hZWXFjg= +github.com/pierrec/lz4/v4 v4.1.29/go.mod h1:EoQMVJgeeEOMsCqCzqFm2O0cJvljX2nGZjcRIPL34O4= github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= @@ -56,25 +71,30 @@ github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY= github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= -github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE= +github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no= github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM= +github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ= +github.com/zeebo/assert v1.3.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0= +github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs= +github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw= +go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg= go.yaml.in/yaml/v4 v4.0.0-rc.4 h1:UP4+v6fFrBIb1l934bDl//mmnoIZEDK0idg1+AIvX5U= go.yaml.in/yaml/v4 v4.0.0-rc.4/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0= -golang.org/x/exp v0.0.0-20220909182711-5c715a9e8561 h1:MDc5xs78ZrZr3HMQugiXOAkSZtfTpbJLDr/lwfgO53E= -golang.org/x/exp v0.0.0-20220909182711-5c715a9e8561/go.mod h1:cyybsKvd6eL0RnXn6p/Grxp8F5bW7iYuBgsNCOHpMYE= +golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU= +golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU= golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs= golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q= -golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= -golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= +golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo= -golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= -golang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU= -golang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A= +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0= +golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w= +gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4= +gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= -gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/internal/config/config.go b/internal/config/config.go index af67d02..e7cbe3b 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -57,14 +57,13 @@ func ValidOutputFormat(s string) bool { // Precedence: flag > OMNI_OUTPUT_FORMAT env > config file > auto(TTY). // An "auto" result from any layer resolves to "human" when isTTY, else "json". func ResolveOutputFormat(flagValue string, isTTY bool) string { - chosen := "" - if flagValue != "" { - chosen = flagValue - } else if v := os.Getenv("OMNI_OUTPUT_FORMAT"); v != "" { - chosen = v - } else if cfg, _ := Load(); cfg != nil && cfg.DefaultOutputFormat != "" { - chosen = cfg.DefaultOutputFormat - } + return FormatFromChoice(ChosenOutputFormat(flagValue), isTTY) +} + +// FormatFromChoice resolves an explicit choice (or none) to the effective +// format. Callers that already hold the choice use this to avoid a second +// read of the config file. +func FormatFromChoice(chosen string, isTTY bool) string { if chosen == "" || chosen == FormatAuto { if isTTY { return FormatHuman @@ -74,6 +73,23 @@ func ResolveOutputFormat(flagValue string, isTTY bool) string { return chosen } +// ChosenOutputFormat reports the format a user actually asked for, by flag, +// env, or config file, before the TTY fallback. It returns "" when nobody +// chose and the format is therefore down to auto-detection — the difference +// between "this output must be JSON" and "this happens to be a pipe". +func ChosenOutputFormat(flagValue string) string { + if flagValue != "" { + return flagValue + } + if v := os.Getenv("OMNI_OUTPUT_FORMAT"); v != "" { + return v + } + if cfg, _ := Load(); cfg != nil && cfg.DefaultOutputFormat != "" { + return cfg.DefaultOutputFormat + } + return "" +} + // ResolvedConfig is the final runtime config after merging flags, env, and file. type ResolvedConfig struct { Token string diff --git a/internal/openapi/generate.go b/internal/openapi/generate.go index e6c5d72..194a7f3 100644 --- a/internal/openapi/generate.go +++ b/internal/openapi/generate.go @@ -251,6 +251,7 @@ type responseInfo struct { ContentType string Description string Schema *base.SchemaProxy // nil when the status declares no body/schema + Streams bool // a text/ndjson query stream is among the declared media types } func extractOperations(pathStr string, item *v3.PathItem, groups map[string][]*operationInfo) { @@ -351,12 +352,21 @@ func buildCommand(op *operationInfo, exec Executor) *cobra.Command { long = "DEPRECATED: " + long } + annotations := map[string]string{} + if props := bodyProperties(op); len(props) > 0 { + annotations[BodyPropsAnnotation] = strings.Join(props, ",") + } + if op.Response != nil && op.Response.Streams { + annotations[StreamAnnotation] = "true" + } + cmd := &cobra.Command{ - Use: use, - Short: short, - Long: long, - Deprecated: deprecatedMsg(op), - Args: cobra.ExactArgs(len(op.PathParams)), + Use: use, + Short: short, + Long: long, + Deprecated: deprecatedMsg(op), + Annotations: annotations, + Args: cobra.ExactArgs(len(op.PathParams)), RunE: func(cmd *cobra.Command, args []string) error { // Flags parsed and args validated: anything that fails from here on // (bad body, HTTP 4xx/5xx) is a runtime error, and dumping the usage @@ -687,6 +697,49 @@ func schemaRequested(cmd *cobra.Command, name string) bool { return err == nil && v } +// BodyPropsAnnotation lists the top-level property names of a command's JSON +// request body, so callers outside this package can ask what a body accepts +// without re-parsing the spec. Read off the spec rather than hard-coded, so an +// endpoint growing a field is covered without a code change. +const BodyPropsAnnotation = "omni/body-props" + +// StreamAnnotation marks a command whose success response is a query stream. +const StreamAnnotation = "omni/stream" + +// ReturnsStream reports whether cmd's success response is a query stream. +func ReturnsStream(cmd *cobra.Command) bool { + return cmd != nil && cmd.Annotations[StreamAnnotation] == "true" +} + +// BodyDeclares reports whether cmd's JSON request body has a top-level +// property called name. +func BodyDeclares(cmd *cobra.Command, name string) bool { + if cmd == nil { + return false + } + for _, p := range strings.Split(cmd.Annotations[BodyPropsAnnotation], ",") { + if p == name { + return true + } + } + return false +} + +func bodyProperties(op *operationInfo) []string { + if op.BodySchema == nil || op.BodyMediaType != "application/json" { + return nil + } + schema := op.BodySchema.Schema() + if schema == nil || schema.Properties == nil { + return nil + } + var props []string + for pair := schema.Properties.First(); pair != nil; pair = pair.Next() { + props = append(props, pair.Key()) + } + return props +} + // requestBodyMediaType returns the media type and definition the CLI will use, // preferring application/json for backward compatibility and otherwise using // the first declared media type. Entries with no schema are skipped, so a @@ -770,6 +823,13 @@ func successResponse(resps *v3.Responses) *responseInfo { info := &responseInfo{Status: bestCode, Description: bestResp.Description} info.ContentType, info.Schema = pickMediaType(bestResp.Content) + if bestResp.Content != nil { + for pair := bestResp.Content.First(); pair != nil; pair = pair.Next() { + if strings.HasPrefix(pair.Key(), "text/ndjson") { + info.Streams = true + } + } + } return info } @@ -909,11 +969,15 @@ func isPluralS(rs []rune, i int) bool { // choosing the host it is sent to. var globalFlagKeys = map[string]string{ // Root persistent flags. - flagLookupKey("profile"): "profile", - flagLookupKey("token"): "token", - flagLookupKey("base-url"): "base-url", - flagLookupKey("compact"): "compact", - flagLookupKey("format"): "format", + flagLookupKey("profile"): "profile", + flagLookupKey("token"): "token", + flagLookupKey("base-url"): "base-url", + flagLookupKey("compact"): "compact", + flagLookupKey("format"): "format", + flagLookupKey("chart"): "chart", + flagLookupKey("chart-value"): "chart-value", + flagLookupKey("chart-rows"): "chart-rows", + flagLookupKey("workbook"): "workbook", // Added by cobra on every command; must stay a bool. flagLookupKey("help"): "help", } diff --git a/internal/openapi/generate_test.go b/internal/openapi/generate_test.go index 3accbd3..37b40c0 100644 --- a/internal/openapi/generate_test.go +++ b/internal/openapi/generate_test.go @@ -2148,3 +2148,52 @@ func TestFirstLine(t *testing.T) { } } } + +// Body properties are read off the spec, so a second endpoint that grows a +// field is covered without a code change here. +func TestBodyDeclares(t *testing.T) { + specData, err := os.ReadFile("../../api/openapi.json") + if err != nil { + t.Skipf("spec not available: %v", err) + } + cmds, err := GenerateCommands(specData, func(APIRequest) error { return nil }) + if err != nil { + t.Fatalf("GenerateCommands: %v", err) + } + find := func(group, name string) *cobra.Command { + for _, g := range cmds { + if g.Name() != group { + continue + } + for _, sub := range g.Commands() { + if sub.Name() == name { + return sub + } + } + } + t.Fatalf("no command %s %s", group, name) + return nil + } + + run := find("query", "run") + for _, prop := range []string{"resultType", "workbookUrl", "planOnly"} { + if !BodyDeclares(run, prop) { + t.Errorf("query run should declare %s", prop) + } + } + if BodyDeclares(run, "nope") { + t.Error("query run should not declare a made-up property") + } + if gen := find("ai", "generate-query"); BodyDeclares(gen, "resultType") { + t.Error("ai generate-query should not declare resultType") + } + if BodyDeclares(nil, "resultType") { + t.Error("a nil command declares nothing") + } + if !ReturnsStream(run) || !ReturnsStream(find("query", "wait")) { + t.Error("query run and query wait return a stream") + } + if ReturnsStream(find("models", "list")) || ReturnsStream(nil) { + t.Error("models list does not return a stream") + } +} diff --git a/internal/output/bench_test.go b/internal/output/bench_test.go new file mode 100644 index 0000000..15974c9 --- /dev/null +++ b/internal/output/bench_test.go @@ -0,0 +1,39 @@ +package output + +import ( + "fmt" + "io" + "testing" + + "github.com/exploreomni/omni-cli/internal/result" +) + +func setOfSize(n int) *result.Set { + set := &result.Set{Columns: []result.Column{ + {Name: "e.country", Label: "Country", IsDimension: true, DataType: "STRING"}, + {Name: "e.sessions", Label: "Sessions", DataType: "NUMBER", Format: "NUMBER_0"}, + {Name: "e.pct", Label: "Pct", DataType: "NUMBER", Format: "percent"}, + }} + for i := 0; i < n; i++ { + set.Rows = append(set.Rows, []any{fmt.Sprintf("country-%d", i%200), int64(i * 7), float64(i%100) / 100}) + } + return set +} + +func BenchmarkRender(b *testing.B) { + for _, n := range []int{100, 1000, 10000} { + set := setOfSize(n) + b.Run(fmt.Sprintf("table/rows=%d", n), func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + ResultTable(io.Discard, set) + } + }) + b.Run(fmt.Sprintf("chart/rows=%d", n), func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + _ = ResultChart(io.Discard, set, ChartOptions{Width: 100, MaxRows: n}) + } + }) + } +} diff --git a/internal/output/chart.go b/internal/output/chart.go new file mode 100644 index 0000000..dc7cacb --- /dev/null +++ b/internal/output/chart.go @@ -0,0 +1,279 @@ +package output + +import ( + "fmt" + "io" + "math" + "strings" + + "github.com/charmbracelet/lipgloss" +) + +// ChartOptions describes a requested chart; zero values mean the defaults. +type ChartOptions struct { + Values []string // measures to draw, by field or label; empty draws every one + Width int + MaxRows int +} + +// barGlyph (▇) leaves a hairline between rows so they don't fuse on tightly +// leaded terminals. +const barGlyph = "▇" + +const DefaultChartRows = 50 + +const ( + defaultChartWidth = 80 + maxLabelWidth = 28 + minLabelWidth = 6 + minBarWidth = 12 +) + +// Beside other bar columns a bar can be short: the value is printed next to +// it, so the bar only has to show proportion. +const minCellBarWidth = 4 + +type chartRow struct { + text string // formatted value + value float64 + present bool +} + +// grid is a chart laid out as rows of labels followed by columns of bars. +type grid struct { + groupLabel string // what pivot values are, over the label columns; "" unpivoted + labelHeaders []string // one per label column + labels [][]string // [row][label column] + cols []gridCol + omittedRows int + omittedCols int +} + +type gridCol struct { + group string // the pivot value over this column; "" unpivoted + header string // the measure's label + scale int // columns with the same scale share one axis (one measure) + items []chartRow +} + +func renderGrid(w io.Writer, g *grid, opts ChartOptions) { + width := opts.Width + if width <= 0 { + width = defaultChartWidth + } + + // Each measure is scaled on its own, across every column it fills. + type bounds struct{ lo, hi float64 } + scales := map[int]bounds{} + valueW := make([]int, len(g.cols)) + textW := make([]int, len(g.cols)) + for c := range g.cols { + col := &g.cols[c] + b := scales[col.scale] + for i := range col.items { + it := &col.items[i] + if !it.present { + it.text = "-" + } else if !math.IsNaN(it.value) && !math.IsInf(it.value, 0) { + b.lo, b.hi = math.Min(b.lo, it.value), math.Max(b.hi, it.value) + } + textW[c] = max(textW[c], lipgloss.Width(it.text)) + } + scales[col.scale] = b + valueW[c] = max(textW[c], lipgloss.Width(col.header)) + } + + labelW := make([]int, len(g.labelHeaders)) + for i, h := range g.labelHeaders { + labelW[i] = lipgloss.Width(h) + for _, row := range g.labels { + labelW[i] = max(labelW[i], lipgloss.Width(row[i])) + } + labelW[i] = min(labelW[i], maxLabelWidth) + } + + n := len(g.cols) + minBar := func() int { + if n == 1 { + return minBarWidth + } + return minCellBarWidth + } + avail := func() int { + used := len(labelW) - 1 + 1 + 2*(n-1) // label gaps, the gap after labels, column gaps + for _, lw := range labelW { + used += lw + } + for c := range n { + used += valueW[c] + 1 + } + return width - used + } + var barW int + for { + // Labels yield to the bars first, so a row never wraps; then columns go. + for avail() < n*minBar() { + widest := 0 + for i := range labelW { + if labelW[i] > labelW[widest] { + widest = i + } + } + if len(labelW) > 0 && labelW[widest] > minLabelWidth { + labelW[widest]-- + continue + } + if n == 1 { + break + } + n-- + g.omittedCols++ + } + barW = max(avail()/n, minBar()) + // A pivot value heading a single column widens it rather than being + // cut off; bars stay one width so a shared scale stays comparable. + grew := false + for c := 0; c < n; c++ { + if spansOne(g.cols, c) { + if need := lipgloss.Width(g.cols[c].group) - 1 - barW; need > valueW[c] { + valueW[c], grew = need, true + } + } + } + if !grew { + break + } + } + + pad := func(s string, cells int) string { + return s + strings.Repeat(" ", max(cells-lipgloss.Width(s), 0)) + } + colW := func(c int) int { + return valueW[c] + 1 + barW + } + labelArea := func(cells []string) string { + parts := make([]string, len(cells)) + for i, s := range cells { + parts[i] = pad(truncateCells(s, labelW[i]), labelW[i]) + } + return strings.Join(parts, " ") + } + labelAreaW := len(labelW) - 1 + for _, lw := range labelW { + labelAreaW += lw + } + gap := func(c int) string { + if c == 0 { + return " " + } + return " " + } + + // A pivot's values head the columns they span, above the measure labels. + if g.groupLabel != "" { + var b strings.Builder + b.WriteString(pad(truncateCells(g.groupLabel, labelAreaW), labelAreaW)) + for c := 0; c < n; { + span := colW(c) + end := c + 1 + for end < n && g.cols[end].group == g.cols[c].group { + span += 2 + colW(end) + end++ + } + b.WriteString(gap(c) + pad(truncateCells(g.cols[c].group, span), span)) + c = end + } + fmt.Fprintln(w, styleDim.Render(strings.TrimRight(b.String(), " "))) + } + + var b strings.Builder + b.WriteString(labelArea(g.labelHeaders)) + for c := range n { + b.WriteString(gap(c)) + b.WriteString(lipgloss.NewStyle().Width(valueW[c]).Align(lipgloss.Right).Render(g.cols[c].header)) + b.WriteString(strings.Repeat(" ", 1+barW)) + } + fmt.Fprintln(w, styleDim.Render(strings.TrimRight(b.String(), " "))) + + for r, row := range g.labels { + var line strings.Builder + line.WriteString(labelArea(row)) + for c := range n { + it := g.cols[c].items[r] + sc := scales[g.cols[c].scale] + last := c == n-1 + line.WriteString(gap(c)) + var drawn string + switch { + case !it.present || math.IsNaN(it.value) || math.IsInf(it.value, 0): + case sc.lo < 0: + drawn = twoSidedBar(it.value, sc.lo, sc.hi, barW) + default: + drawn = styleBar.Render(blocks(it.value, sc.hi, barW)) + } + line.WriteString(lipgloss.NewStyle().Width(valueW[c]).Align(lipgloss.Right).Render(it.text) + " ") + if !last { + drawn = pad(drawn, barW) + } + line.WriteString(drawn) + } + fmt.Fprintln(w, strings.TrimRight(line.String(), " ")) + } + + var notes []string + if g.omittedRows > 0 { + notes = append(notes, fmt.Sprintf("%d more row%s", g.omittedRows, plural(g.omittedRows))) + } + if g.omittedCols > 0 { + notes = append(notes, fmt.Sprintf("%d more column%s", g.omittedCols, plural(g.omittedCols))) + } + if len(notes) > 0 { + fmt.Fprintln(w, styleDim.Render("… and "+strings.Join(notes, ", "))) + } +} + +func spansOne(cols []gridCol, c int) bool { + g := cols[c].group + return g != "" && (c == 0 || cols[c-1].group != g) && (c == len(cols)-1 || cols[c+1].group != g) +} + +// blocks renders v/scale of width; a non-zero value is always at least one cell. +func blocks(v, scale float64, width int) string { + if scale <= 0 || v <= 0 || width <= 0 { + return "" + } + n := int(math.Round((v / scale) * float64(width))) + return strings.Repeat(barGlyph, min(max(n, 1), width)) +} + +// twoSidedBar draws around a zero axis. +func twoSidedBar(v, lo, hi float64, width int) string { + span := hi - lo + if span <= 0 { + return "" + } + usable := width - 1 + if usable < 2 { + return "" + } + left := int(math.Round((-lo / span) * float64(usable))) + left = min(max(left, 1), usable-1) + right := usable - left + + if v < 0 { + b := blocks(-v, -lo, left) + return strings.Repeat(" ", left-lipgloss.Width(b)) + styleNeg.Render(b) + styleDim.Render("│") + } + return strings.Repeat(" ", left) + styleDim.Render("│") + styleBar.Render(blocks(v, hi, right)) +} + +func ChartLink(w io.Writer, url string) { + fmt.Fprintf(w, "%s %s\n", styleDim.Render("Open in Omni:"), singleLine(url)) +} + +func plural(n int) string { + if n == 1 { + return "" + } + return "s" +} diff --git a/internal/output/chart_test.go b/internal/output/chart_test.go new file mode 100644 index 0000000..5661040 --- /dev/null +++ b/internal/output/chart_test.go @@ -0,0 +1,492 @@ +package output + +import ( + "bytes" + "math" + "math/big" + "strings" + "testing" + + "github.com/charmbracelet/lipgloss" + "github.com/exploreomni/omni-cli/internal/result" +) + +func col(name, label string, dim bool, format string) result.Column { + return result.Column{Name: name, Label: label, IsDimension: dim, DataType: "NUMBER", Format: format} +} + +// The Ireland/US query as the stream decodes it. +func sessionsSet() *result.Set { + return &result.Set{ + Columns: []result.Column{ + {Name: "events_ext.country", Label: "Country", IsDimension: true, DataType: "STRING"}, + col("events_ext.sessions", "Sessions", false, "NUMBER_0"), + col("events_ext.engaged_sessions_percent", "Engaged Sessions %", false, "percent"), + }, + Rows: [][]any{ + {"United States", int64(12526), 0.3992495609133003}, + {"Ireland", int64(838), 0.4486873508353222}, + }, + } +} + +// headerIs reports whether the first line carries the two column headers, +// label left and value right, in that order. +func headerIs(out, label, value string) bool { + first, _, _ := strings.Cut(out, "\n") + fields := strings.Fields(first) + return len(fields) >= 2 && strings.HasPrefix(first, label) && strings.HasSuffix(strings.TrimRight(first, " "), value) +} + +// The headers sit over their columns: the label header flush left, the +// value header right-aligned to the numbers beneath it. +func TestChart_HeadersAlignToColumns(t *testing.T) { + out := chart(t, sessionsSet(), ChartOptions{}) + lines := strings.Split(out, "\n") + header, row := lines[0], lines[1] + hEnd := strings.Index(header, "Sessions") + len("Sessions") + vEnd := strings.Index(row, "12,526") + len("12,526") + if lipgloss.Width(header[:hEnd]) != lipgloss.Width(row[:vEnd]) { + t.Errorf("value header should end where the values end:\n%s", out) + } +} + +func chart(t *testing.T, set *result.Set, opts ChartOptions) string { + t.Helper() + var buf bytes.Buffer + if opts.Width == 0 { + opts.Width = 60 + } + if err := ResultChart(&buf, set, opts); err != nil { + t.Fatalf("ResultChart: %v", err) + } + return buf.String() +} + +func chartErr(t *testing.T, set *result.Set, opts ChartOptions) string { + t.Helper() + var buf bytes.Buffer + err := ResultChart(&buf, set, opts) + if err == nil { + t.Fatalf("expected an error, got output:\n%s", buf.String()) + } + return err.Error() +} + +// The model says which columns are dimensions and which measures; the chart +// doesn't guess. Every measure gets bars, in query order. +func TestChart_ColumnsFromModel(t *testing.T) { + out := chart(t, sessionsSet(), ChartOptions{Width: 80}) + if !headerIs(out, "Country", "Engaged Sessions %") { + t.Fatalf("expected the dimension and every measure, got:\n%s", out) + } + header, _, _ := strings.Cut(out, "\n") + if i := strings.Index(header, "Sessions"); i < 0 || i > strings.Index(header, "Engaged") { + t.Errorf("measures should keep query order:\n%s", out) + } + if !strings.Contains(out, "12,526") { + t.Errorf("NUMBER_0 should group digits:\n%s", out) + } +} + +func TestChart_ValueFormattedByModel(t *testing.T) { + out := chart(t, sessionsSet(), ChartOptions{Values: []string{"engaged_sessions_percent"}}) + if !headerIs(out, "Country", "Engaged Sessions %") { + t.Fatalf("expected the percent measure, got:\n%s", out) + } + if !strings.Contains(out, "39.9%") || !strings.Contains(out, "44.9%") { + t.Errorf("percent format should apply:\n%s", out) + } +} + +// Several values draw in the order given, each once. +func TestChart_SeveralValues(t *testing.T) { + out := chart(t, sessionsSet(), ChartOptions{Values: []string{"Engaged Sessions %", "sessions", "events_ext.sessions"}, Width: 80}) + if !headerIs(out, "Country", "Sessions") { + t.Fatalf("expected the flag's order, ending with Sessions:\n%s", out) + } + header, _, _ := strings.Cut(out, "\n") + if strings.Count(header, "Sessions") != 2 { + t.Errorf("a column named twice should draw once:\n%s", out) + } +} + +// Rows are keyed by field name, the model by label; either spelling should +// find the column. +func TestChart_ColumnSpellings(t *testing.T) { + for _, spelling := range []string{ + "Engaged Sessions %", + "engaged_sessions_percent", + "events_ext.engaged_sessions_percent", + "ENGAGED SESSIONS PERCENT", + } { + out := chart(t, sessionsSet(), ChartOptions{Values: []string{spelling}}) + if !headerIs(out, "Country", "Engaged Sessions %") { + t.Errorf("%q did not select the column:\n%s", spelling, out) + } + } + // A bare field name finds a column whose label reads differently. + if i, ok := matchColumn(pipelineSet().Columns, "count"); !ok || i != 3 { + t.Errorf(`"count" should find deals.count ("Deals Count"), got %d %v`, i, ok) + } + if _, ok := matchColumn(sessionsSet().Columns, "session"); ok { + t.Error("a near miss should not match") + } +} + +func TestChart_BarLengthsScaleToMax(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "r", Label: "Region", IsDimension: true}, col("v", "Revenue", false, "")}, + Rows: [][]any{{"east", int64(1000)}, {"west", int64(500)}, {"north", int64(0)}}, + } + lines := strings.Split(strings.TrimRight(chart(t, set, ChartOptions{}), "\n"), "\n") + full := strings.Count(lines[1], "▇") + half := strings.Count(lines[2], "▇") + if full == 0 || half == 0 { + t.Fatalf("expected drawn bars:\n%s", strings.Join(lines, "\n")) + } + if got, want := half, full/2; got < want-1 || got > want+1 { + t.Errorf("half-value bar is %d blocks, expected about %d", got, want) + } + if strings.Contains(lines[3], "▇") { + t.Errorf("zero row should have no bar: %q", lines[3]) + } +} + +// A tiny value still shows as at least one cell, rounded to whole cells. +func TestChart_TinyValueDrawsOneCell(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "r", Label: "Region", IsDimension: true}, col("v", "Revenue", false, "")}, + Rows: [][]any{{"east", int64(1000)}, {"west", int64(1)}}, + } + lines := strings.Split(strings.TrimRight(chart(t, set, ChartOptions{}), "\n"), "\n") + if got := strings.Count(lines[2], "▇"); got != 1 { + t.Errorf("a tiny value should draw one cell, got %d:\n%s", got, strings.Join(lines, "\n")) + } +} + +func TestChart_NonFiniteValueDrawsNoBar(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "r", Label: "Region", IsDimension: true}, col("v", "Revenue", false, "")}, + Rows: [][]any{{"east", 100.0}, {"west", math.NaN()}, {"north", math.Inf(1)}}, + } + lines := strings.Split(strings.TrimRight(chart(t, set, ChartOptions{}), "\n"), "\n") + for _, line := range lines[2:] { + if strings.Contains(line, "▇") { + t.Errorf("a non-finite value should draw no bar: %q", line) + } + if !strings.Contains(line, "NaN") && !strings.Contains(line, "Inf") { + t.Errorf("the value's text should still show: %q", line) + } + } +} + +func TestChart_DecimalValues(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "r", Label: "Region", IsDimension: true}, col("v", "Revenue", false, "number_2")}, + Rows: [][]any{ + {"east", result.Decimal{Coef: big.NewInt(100000), Scale: 2}}, + {"west", result.Decimal{Coef: big.NewInt(50000), Scale: 2}}, + }, + } + lines := strings.Split(strings.TrimRight(chart(t, set, ChartOptions{Width: 40}), "\n"), "\n") + if !strings.Contains(lines[1], "1,000.00") || !strings.Contains(lines[2], "500.00") { + t.Fatalf("decimals should render in the model format:\n%s", strings.Join(lines, "\n")) + } + if east, west := barCells(lines[1]), barCells(lines[2]); east != 2*west { + t.Errorf("bars should scale to the decimal values, got %d and %d", east, west) + } +} + +func TestChart_NullValueRendersAsDash(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "a", Label: "A", IsDimension: true}, col("b", "B", false, "")}, + Rows: [][]any{{"x", int64(5)}, {"y", nil}}, + } + out := chart(t, set, ChartOptions{}) + if !strings.Contains(out, "\ny -\n") { + t.Errorf("expected a dash for the null row:\n%s", out) + } +} + +func TestChart_MixedSignsDrawAnAxis(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "m", Label: "Month", IsDimension: true}, col("d", "Change", false, "")}, + Rows: [][]any{{"jan", 400.0}, {"feb", -200.0}}, + } + if out := chart(t, set, ChartOptions{}); strings.Count(out, "│") != 2 { + t.Errorf("expected an axis on each bar:\n%s", out) + } +} + +// Rows have to fit the terminal: the label column yields to the bar rather +// than pushing the line past the width and wrapping. +func TestChart_RowsFitTheWidth(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "c", Label: "Category", IsDimension: true}, col("t", "Total", false, "NUMBER_2")}, + Rows: [][]any{{"Fashion Hoodies & Sweatshirts Extra Long", 1602513.81}, {"Accessories", 955617.3}}, + } + for _, width := range []int{40, 50, 80, 160} { + out := chart(t, set, ChartOptions{Width: width}) + for _, line := range strings.Split(strings.TrimRight(out, "\n"), "\n") { + if got := lipgloss.Width(line); got > width { + t.Errorf("width %d: line is %d cells: %q", width, got, line) + } + } + } +} + +func TestChart_WideRuneLabelsFitTheWidth(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "n", Label: "名前", IsDimension: true}, col("v", "値", false, "")}, + Rows: [][]any{{"東京都渋谷区神宮前一丁目二番三号", int64(100)}, {"大阪", int64(50)}}, + } + out := chart(t, set, ChartOptions{Width: 60}) + for _, line := range strings.Split(strings.TrimRight(out, "\n"), "\n") { + if got := lipgloss.Width(line); got > 60 { + t.Errorf("line is %d cells: %q", got, line) + } + } +} + +func TestChart_RowCapReportsTheRemainder(t *testing.T) { + set := &result.Set{Columns: []result.Column{{Name: "a", Label: "A", IsDimension: true}, col("b", "B", false, "")}} + for range 10 { + set.Rows = append(set.Rows, []any{"x", int64(1)}) + } + if out := chart(t, set, ChartOptions{MaxRows: 4}); !strings.Contains(out, "… and 6 more rows") { + t.Errorf("expected a remainder note:\n%s", out) + } +} + +// Two dimensions and no measure: the first numeric column stands in, and +// only a result with nothing numeric is refused. +func TestChart_NoMeasure(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{ + {Name: "a", Label: "A", IsDimension: true}, + {Name: "year", Label: "Year", IsDimension: true, DataType: "NUMBER"}, + }, + Rows: [][]any{{"x", int64(2026)}}, + } + // With no measure, the first numeric column stands in. + if out := chart(t, set, ChartOptions{}); !headerIs(out, "A", "Year") { + t.Errorf("expected the numeric dimension to be plotted:\n%s", out) + } + set.Columns[1].DataType = "STRING" + set.Rows = [][]any{{"x", "twenty"}} + if got := chartErr(t, set, ChartOptions{}); !strings.Contains(got, "nothing numeric") { + t.Errorf("unexpected error: %s", got) + } +} + +func TestChart_Errors(t *testing.T) { + set := sessionsSet() + tests := []struct { + name string + opts ChartOptions + want string + }{ + {"unknown value column", ChartOptions{Values: []string{"nope"}}, "is not a column"}, + {"non-numeric value column", ChartOptions{Values: []string{"sessions", "country"}}, "holds no numbers"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + if got := chartErr(t, set, tc.opts); !strings.Contains(got, tc.want) { + t.Errorf("error %q does not mention %q", got, tc.want) + } + }) + } +} + +func TestChart_EmptySet(t *testing.T) { + set := &result.Set{Columns: sessionsSet().Columns} + if out := chart(t, set, ChartOptions{}); !strings.Contains(out, "No results.") { + t.Errorf("expected the empty-result line, got %q", out) + } +} + +func TestResultTable(t *testing.T) { + var buf bytes.Buffer + ResultTable(&buf, sessionsSet()) + out := buf.String() + for _, want := range []string{"Country", "Sessions", "Engaged Sessions %", "12,526", "39.9%", "44.9%"} { + if !strings.Contains(out, want) { + t.Errorf("table missing %q:\n%s", want, out) + } + } + // Query order, not alphabetical: Country before Engaged before Sessions + // would be alphabetical; the model's order has Sessions second. + if strings.Index(out, "Sessions") > strings.Index(out, "Engaged") { + t.Errorf("columns should keep query order:\n%s", out) + } +} + +// Region × stage with two measures, as the stream decodes it. +func pipelineSet() *result.Set { + return &result.Set{ + Columns: []result.Column{ + {Name: "deals.region", Label: "Region", IsDimension: true, DataType: "STRING"}, + {Name: "deals.stage", Label: "Stage", IsDimension: true, DataType: "STRING"}, + col("deals.total_amount", "Total amount", false, "currency_0"), + col("deals.count", "Deals Count", false, ""), + }, + Rows: [][]any{ + {"AMER", "Closed Lost", int64(13966500), int64(223)}, + {"AMER", "Negotiation", int64(167500), int64(3)}, + {"AMER", "Closed Won", int64(3903000), int64(56)}, + {"EMEA", "Closed Lost", int64(8482500), int64(125)}, + {"EMEA", "Closed Won", int64(1949500), int64(36)}, + }, + } +} + +func pivoted() *result.Set { + set := pipelineSet() + set.Pivots = []string{"deals.stage"} + return set +} + +func barCells(line string) int { + return strings.Count(line, "▇") +} + +// Two dimensions label each bar together, so rows don't read as repeats of +// the first; each measure is scaled to its own maximum. +func TestChart_TwoDimensionsTwoMeasures(t *testing.T) { + out := chart(t, pipelineSet(), ChartOptions{Width: 100}) + lines := strings.Split(strings.TrimRight(out, "\n"), "\n") + if f := strings.Fields(lines[0]); len(f) < 2 || f[0] != "Region" || f[1] != "Stage" { + t.Fatalf("expected both dimensions as columns:\n%s", out) + } + if !strings.Contains(lines[1], "AMER") || !strings.Contains(lines[1], "Closed Lost") { + t.Errorf("row should carry both dimension values:\n%s", out) + } + // Closed Lost is the max of both measures: its two bars are both full. + amount, count := strings.Split(lines[1], "$13,966,500")[1], strings.Split(lines[1], "223")[1] + if barCells(amount)-barCells(count) != barCells(strings.Split(amount, "223")[0]) { + t.Errorf("each measure should fill its own column at its max:\n%s", out) + } + for _, line := range lines { + if got := lipgloss.Width(line); got > 100 { + t.Errorf("line is %d cells: %q", got, line) + } + } +} + +// A pivot spreads the measure across its values like the Omni app's bar +// table: the pivot values head the columns, and they share one scale. +func TestChart_Pivot(t *testing.T) { + out := chart(t, pivoted(), ChartOptions{Values: []string{"Total amount"}, Width: 120}) + lines := strings.Split(strings.TrimRight(out, "\n"), "\n") + for _, want := range []string{"Stage", "Closed Lost", "Negotiation", "Closed Won"} { + if !strings.Contains(lines[0], want) { + t.Errorf("group header missing %q:\n%s", want, out) + } + } + if !strings.HasPrefix(lines[1], "Region") || strings.Count(lines[1], "Total amount") != 1 { + t.Errorf("a lone measure should be named once:\n%s", out) + } + if narrow := chart(t, pivoted(), ChartOptions{Values: []string{"count"}, Width: 60}); strings.Contains(strings.Split(narrow, "\n")[0], "…") { + t.Errorf("pivot values should widen their column, not be cut off:\n%s", narrow) + } + both := chart(t, pivoted(), ChartOptions{Width: 160}) + if header := strings.Split(both, "\n")[1]; strings.Count(header, "Total amount") < 2 || strings.Count(header, "Deals Count") < 2 { + t.Errorf("with two measures each column should say which:\n%s", both) + } + // Naming both measures draws what no flag draws. + if picked := chart(t, pivoted(), ChartOptions{Values: []string{"Total amount", "count"}, Width: 160}); picked != both { + t.Errorf("naming every measure should match the default:\n%s\nvs\n%s", picked, both) + } + if len(lines) != 4 { + t.Fatalf("expected two header lines and a row per region:\n%s", out) + } + if !strings.HasPrefix(lines[3], "EMEA") || !strings.Contains(lines[3], " - ") { + t.Errorf("EMEA has no Negotiation deals and should show a dash:\n%s", out) + } + // Shared scale: AMER Closed Lost ($13.97M) is the longest bar, and EMEA + // Closed Lost ($8.48M) is shorter than it though it's EMEA's largest. + full := strings.Count(strings.Split(lines[2], "$3,903,000")[0], "▇") + emea := strings.Count(strings.Split(lines[3], "$1,949,500")[0], "▇") + if emea >= full || emea == 0 { + t.Errorf("pivot columns should share the measure's scale (full %d, emea %d):\n%s", full, emea, out) + } +} + +// A pivot too wide for the terminal drops trailing columns and says so. +func TestChart_PivotFitsTheWidth(t *testing.T) { + out := chart(t, pivoted(), ChartOptions{Width: 50}) + for _, line := range strings.Split(strings.TrimRight(out, "\n"), "\n") { + if got := lipgloss.Width(line); got > 50 { + t.Errorf("line is %d cells: %q", got, line) + } + } + if !strings.Contains(out, "more column") { + t.Errorf("expected a note about dropped columns:\n%s", out) + } +} + +func TestChart_PivotErrors(t *testing.T) { + for _, tc := range []struct { + opts ChartOptions + want string + }{ + {ChartOptions{Values: []string{"region"}}, "not a measure"}, + {ChartOptions{Values: []string{"count", "nope"}}, "is not a column"}, + } { + if got := chartErr(t, pivoted(), tc.opts); !strings.Contains(got, tc.want) { + t.Errorf("%+v: error %q does not mention %q", tc.opts, got, tc.want) + } + } +} + +func TestResultTable_Pivot(t *testing.T) { + var buf bytes.Buffer + ResultTable(&buf, pivoted()) + out := buf.String() + lines := strings.Split(out, "\n") + if strings.Count(out, "├") != 1 { + t.Errorf("expected one rule, under the header:\n%s", out) + } + // Two header lines: pivot values over measure labels. + if !strings.Contains(lines[1], "Stage") || !strings.Contains(lines[1], "Closed Lost") || !strings.Contains(lines[2], "Region") || strings.Count(lines[2], "Deals Count") != 3 { + t.Fatalf("expected pivot values over measure labels:\n%s", out) + } + if strings.Count(out, "AMER") != 1 || strings.Count(out, "EMEA") != 1 { + t.Errorf("expected one row per region:\n%s", out) + } +} + +func TestResult_StripsControlCharacters(t *testing.T) { + set := &result.Set{ + Columns: []result.Column{{Name: "r", Label: "Region\x1b[31m", IsDimension: true}, col("v", "Total", false, "0\"\x1b]0;changed\a\"")}, + Rows: [][]any{{"east\x1b]0;pwned\x07\nwest", int64(5)}}, + } + var table, link bytes.Buffer + ResultTable(&table, set) + ChartLink(&link, "https://x/e/1\x1b[2J") + for _, out := range []string{table.String(), chart(t, set, ChartOptions{}), link.String()} { + if strings.ContainsAny(out, "\x1b\x07") { + t.Errorf("control characters reached the output:\n%q", out) + } + } + // A newline in a value would break the layout; it reads as a space. + if out := chart(t, set, ChartOptions{}); strings.Count(out, "\n") != 2 { + t.Errorf("expected a header and one row:\n%q", out) + } +} + +// Three measures, and a column limit that drops two pivot values: six +// columns are missing, not two. +func TestChart_PivotOmittedCountsColumns(t *testing.T) { + set := pivoted() + set.Columns = append(set.Columns, col("deals.won", "Won", false, "")) + for i := range set.Rows { + set.Rows[i] = append(set.Rows[i], int64(1)) + } + set.ColumnLimit = 1 + out := chart(t, set, ChartOptions{Width: 160}) + if !strings.Contains(out, "… and 6 more columns") { + t.Errorf("expected six omitted columns:\n%s", out) + } +} diff --git a/internal/output/format.go b/internal/output/format.go new file mode 100644 index 0000000..76171bb --- /dev/null +++ b/internal/output/format.go @@ -0,0 +1,636 @@ +package output + +import ( + "fmt" + "math" + "regexp" + "strconv" + "strings" + "time" + + "github.com/exploreomni/omni-cli/internal/result" +) + +// numFormat is a decoded model format: a documented name or an Excel-style pattern. +type numFormat struct { + kind string // number, percent, id, big, billions, millions, thousands, currency, accounting, financial, pattern + decimals int // -1 when the format leaves it open + // pattern only: decimals past this many are dropped when they're zeros (#) + minDecimals int + minInt int // pattern only: integer digits always shown (00000) + symbol string // currency family only + compact bool // bigcurrency, bigaccounting, bigfinancial + + // pattern only + prefix, suffix string + group, scale bool + divide float64 // trailing commas: ÷1,000 each + exponent bool // 0.00E+00 + negative, zero *numFormat +} + +// namedFormat: a numeric kind, or a currency category with optional big and +// currency-code prefixes (bigusdcurrency_2), plus an optional _. +var namedFormat = regexp.MustCompile(`(?i)^(?:(number|percent|id|big|billions|millions|thousands)|(big)?(usd|eur|gbp|aud|jpy|brl)?(currency|accounting|financial))(?:_(\d))?$`) + +var currencySymbols = map[string]string{"usd": "$", "eur": "€", "gbp": "£", "aud": "A$", "jpy": "¥", "brl": "R$"} + +func parseFormat(f string) (numFormat, bool) { + f = strings.TrimSpace(f) + m := namedFormat.FindStringSubmatch(f) + if m == nil { + return parsePattern(f) + } + nf := numFormat{decimals: -1, symbol: "$"} + if m[1] != "" { + nf.kind = strings.ToLower(m[1]) + } else { + nf.kind = strings.ToLower(m[4]) + nf.compact = m[2] != "" + if code := strings.ToLower(m[3]); code != "" { + nf.symbol = currencySymbols[code] + } + } + if m[5] != "" { + nf.decimals = int(m[5][0] - '0') + } + return nf, true +} + +// parsePattern reads an Excel-style pattern: quoted/escaped literals, a bare +// % scales by 100, trailing commas divide by 1000, E+00, and pos;neg;zero +// sections. {{field}} references need data the result lacks and are declined. +func parsePattern(p string) (numFormat, bool) { + if p == "" || strings.Contains(p, "{{") { + return numFormat{}, false + } + sections := splitSections(p) + nf, ok := parseSection(sections[0]) + if !ok { + return numFormat{}, false + } + if len(sections) > 1 { + if neg, ok := parseSection(sections[1]); ok { + nf.negative = &neg + } + } + if len(sections) > 2 { + if zero, ok := parseSection(sections[2]); ok { + nf.zero = &zero + } + } + return nf, true +} + +func splitSections(p string) []string { + var out []string + var cur strings.Builder + quoted := false + for i := 0; i < len(p); i++ { + switch { + case p[i] == '"': + quoted = !quoted + case p[i] == ';' && !quoted: + out = append(out, cur.String()) + cur.Reset() + continue + } + cur.WriteByte(p[i]) + } + return append(out, cur.String()) +} + +func parseSection(p string) (numFormat, bool) { + nf := numFormat{kind: "pattern", divide: 1} + var core strings.Builder + inCore, afterCore := false, false + for i := 0; i < len(p); { + c := p[i] + switch { + case c == '"': + end := strings.IndexByte(p[i+1:], '"') + if end < 0 { + return numFormat{}, false + } + nf.addLiteral(p[i+1:i+1+end], inCore) + i += end + 2 + case c == '\\' && i+1 < len(p): + nf.addLiteral(p[i+1:i+2], inCore) + i += 2 + case c == '#' || c == '0' || c == '.': + if afterCore { + return numFormat{}, false + } + inCore = true + core.WriteByte(c) + i++ + case c == ',': + if !inCore || afterCore { + return numFormat{}, false + } + if j := i + 1; j < len(p) && (p[j] == '#' || p[j] == '0') { + nf.group = true + } else { + nf.divide *= 1000 + } + i++ + case c == 'E' && strings.HasPrefix(p[i:], "E+00"): + nf.exponent = true + afterCore = true + i += 4 + case c == '%': + nf.scale = true + nf.addLiteral("%", inCore) + i++ + case c == '$' || c == ' ' || c == '-' || c == '+' || c == '(' || c == ')': + nf.addLiteral(string(c), inCore) + i++ + case currencyPrefix(p[i:]) != "": + sym := currencyPrefix(p[i:]) + nf.addLiteral(sym, inCore) + i += len(sym) + default: + return numFormat{}, false + } + if inCore && !(c == '#' || c == '0' || c == '.' || c == ',') { + afterCore = true + } + } + digits := core.String() + if digits == "" { + return numFormat{}, false + } + intDigits := digits + if dot := strings.IndexByte(digits, '.'); dot >= 0 { + // 0 is a digit always shown, # one shown only when it isn't a trailing zero. + nf.minDecimals = strings.Count(digits[dot+1:], "0") + nf.decimals = nf.minDecimals + strings.Count(digits[dot+1:], "#") + intDigits = digits[:dot] + } + nf.minInt = strings.Count(intDigits, "0") + return nf, true +} + +func currencyPrefix(s string) string { + for _, sym := range []string{"€", "£", "¥"} { + if strings.HasPrefix(s, sym) { + return sym + } + } + return "" +} + +func (nf *numFormat) addLiteral(s string, afterCore bool) { + if afterCore { + nf.suffix += s + } else { + nf.prefix += s + } +} + +// FormatValue renders one cell in its column's model format. +func FormatValue(v any, col result.Column) string { + return singleLine(formatValue(v, col)) +} + +func formatValue(v any, col result.Column) string { + switch x := v.(type) { + case nil: + return "-" + case time.Time: + if strings.Contains(col.Format, "%") { + return strftime(x, col.Format) + } + return formatTime(x, col.DataType) + case bool: + if x { + return "true" + } + return "false" + case string: + x = singleLine(x) + if x == "" { + return "-" + } + // Temporal dimensions arrive as ISO text; a strftime format is ours to apply. + if strings.Contains(col.Format, "%") && isTemporal(col.DataType) { + if t, ok := parseISO(x); ok { + return strftime(t, col.Format) + } + } + return x + } + nf, formatted := parseFormat(col.Format) + if ex, ok := asExact(v); ok { + if !formatted { + return ex.plain() + } + if s, ok := nf.renderExact(ex); ok { + return s + } + } + f, ok := asFloat(v) + if !ok { + return singleLine(fmt.Sprint(v)) + } + if !formatted { + return formatNumber(f) + } + return nf.render(f) +} + +// exact is an integer or decimal kept as digits, never rounded through float64. +type exact struct { + neg bool + digits string // the unsigned coefficient + scale int +} + +func asExact(v any) (exact, bool) { + var s string + scale := 0 + switch x := v.(type) { + case int64: + s = strconv.FormatInt(x, 10) + case int: + s = strconv.Itoa(x) + case result.Decimal: + s, scale = x.Coef.String(), int(x.Scale) + default: + return exact{}, false + } + if strings.HasPrefix(s, "-") { + return exact{neg: true, digits: s[1:], scale: scale}, true + } + return exact{digits: s, scale: scale}, true +} + +func (e exact) isZero() bool { + return strings.Trim(e.digits, "0") == "" +} + +// fixed rounds half away from zero; neg is false when that rounds to zero. +func (e exact) fixed(decimals, minInt int, group bool) (body string, neg bool) { + digits := e.digits + if decimals < e.scale { + cut := len(digits) - (e.scale - decimals) + roundUp := cut >= 0 && digits[cut] >= '5' + digits = digits[:max(cut, 0)] + if roundUp { + digits = incrementDigits(digits) + } + } else { + digits += strings.Repeat("0", decimals-e.scale) + } + if len(digits) <= decimals { + digits = strings.Repeat("0", decimals-len(digits)+1) + digits + } + intPart, frac := digits[:len(digits)-decimals], digits[len(digits)-decimals:] + intPart = strings.TrimLeft(intPart, "0") + if len(intPart) < max(minInt, 1) { + intPart = strings.Repeat("0", max(minInt, 1)-len(intPart)) + intPart + } + if group { + intPart = groupDigits(intPart) + } + if frac != "" { + intPart += "." + frac + } + return intPart, e.neg && strings.Trim(digits, "0") != "" +} + +func incrementDigits(d string) string { + b := []byte(d) + for i := len(b) - 1; i >= 0; i-- { + if b[i] < '9' { + b[i]++ + return string(b) + } + b[i] = '0' + } + return "1" + string(b) +} + +// plain matches formatNumber: trailing zeros dropped, grouped from five digits. +func (e exact) plain() string { + body, neg := e.fixed(e.scale, 1, false) + intPart, frac := body, "" + if i := strings.IndexByte(body, '.'); i >= 0 { + intPart, frac = body[:i], strings.TrimRight(body[i:], "0") + if frac == "." { + frac = "" + } + } + if len(intPart) >= 5 { + intPart = groupDigits(intPart) + } + return signed(intPart+frac, neg) +} + +func signed(s string, neg bool) string { + if neg { + return "-" + s + } + return s +} + +func (nf numFormat) renderExact(e exact) (string, bool) { + switch nf.kind { + case "id": + return signed(e.fixed(0, 1, false)), true + case "number": + return signed(e.fixed(defaultDecimals(nf, 2), 1, true)), true + case "currency", "accounting", "financial": + if nf.compact { + return "", false + } + body, neg := e.fixed(defaultDecimals(nf, 2), 1, true) + if nf.kind == "financial" { + if neg { + body = "(" + body + ")" + } + return body, true + } + if neg { + if nf.kind == "currency" { + return "-" + nf.symbol + body, true + } + body = "(" + body + ")" + } + return nf.symbol + body, true + case "pattern": + sign := false + if e.neg && !e.isZero() { + if nf.negative != nil { + nf = *nf.negative + } else { + sign = true + } + } else if e.isZero() && nf.zero != nil { + nf = *nf.zero + } + if nf.scale || nf.divide != 1 || nf.exponent { + return "", false + } + body, neg := e.fixed(nf.decimals, nf.minInt, nf.group) + if nf.minDecimals < nf.decimals { + body = trimDecimals(body, nf.minDecimals) + } + return signed(nf.prefix+body+nf.suffix, sign && neg), true + } + return "", false +} + +func (nf numFormat) render(f float64) string { + if math.IsNaN(f) || math.IsInf(f, 0) { + return fmt.Sprintf("%g", f) + } + abs := math.Abs(f) + switch nf.kind { + case "percent": + return fixed(f*100, defaultDecimals(nf, 1), true) + "%" + case "id": + return fixed(f, 0, false) + case "big": + return bigNumber(f, defaultDecimals(nf, 2)) + case "billions": + return fixed(f/1e9, defaultDecimals(nf, 2), true) + "B" + case "millions": + return fixed(f/1e6, defaultDecimals(nf, 1), true) + "M" + case "thousands": + return fixed(f/1e3, defaultDecimals(nf, 2), true) + "K" + case "currency", "accounting", "financial": + d := defaultDecimals(nf, 2) + var body string + if nf.compact { + body = bigNumber(abs, d) + } else { + body = fixed(abs, d, true) + } + switch nf.kind { + case "currency": + if f < 0 { + return "-" + nf.symbol + body + } + return nf.symbol + body + case "accounting": + if f < 0 { + return nf.symbol + "(" + body + ")" + } + return nf.symbol + body + default: // financial: parentheses, no mark + if f < 0 { + return "(" + body + ")" + } + return body + } + case "pattern": + if f < 0 && nf.negative != nil { + return nf.negative.renderSection(abs) + } + if f == 0 && nf.zero != nil { + return nf.zero.renderSection(0) + } + s := nf.renderSection(abs) + if f < 0 { + return "-" + s + } + return s + } + // number + if nf.decimals < 0 { + return fixed(f, 2, true) + } + return fixed(f, nf.decimals, true) +} + +func (nf numFormat) renderSection(abs float64) string { + if nf.scale { + abs *= 100 + } + abs /= nf.divide + var s string + if nf.exponent { + s = strings.ToUpper(fmt.Sprintf("%.*e", nf.decimals, abs)) + } else { + s = padInteger(fixed(abs, nf.decimals, false), nf.minInt) + if nf.group { + intPart, frac, _ := strings.Cut(s, ".") + s = groupDigits(intPart) + if frac != "" { + s += "." + frac + } + } + if nf.minDecimals < nf.decimals { + s = trimDecimals(s, nf.minDecimals) + } + } + return nf.prefix + s + nf.suffix +} + +func defaultDecimals(nf numFormat, d int) int { + if nf.decimals < 0 { + return d + } + return nf.decimals +} + +// fixed renders f with exactly d decimals, optionally grouping thousands. +func fixed(f float64, d int, group bool) string { + s := fmt.Sprintf("%.*f", d, f) + sign := "" + if strings.HasPrefix(s, "-") { + sign, s = "-", s[1:] + } + intPart, frac := s, "" + if i := strings.IndexByte(s, '.'); i >= 0 { + intPart, frac = s[:i], s[i:] + } + if group { + intPart = groupDigits(intPart) + } + return sign + intPart + frac +} + +func padInteger(s string, minInt int) string { + intLen := strings.IndexByte(s, '.') + if intLen < 0 { + intLen = len(s) + } + if intLen >= minInt { + return s + } + return strings.Repeat("0", minInt-intLen) + s +} + +func groupDigits(digits string) string { + if len(digits) <= 3 { + return digits + } + var b strings.Builder + for i := range digits { + if i > 0 && (len(digits)-i)%3 == 0 { + b.WriteByte(',') + } + b.WriteByte(digits[i]) + } + return b.String() +} + +// trimDecimals drops trailing zero decimals from a fixed rendering, keeping keep of them. +func trimDecimals(s string, keep int) string { + dot := strings.IndexByte(s, '.') + if dot < 0 { + return s + } + end := len(s) + for end > dot+1+keep && s[end-1] == '0' { + end-- + } + if end == dot+1 { + end = dot + } + return s[:end] +} + +// bigNumber picks the unit after rounding, so 999,999.9 reads 1.00M, not 1000.00K. +func bigNumber(f float64, d int) string { + abs := math.Abs(f) + pow := math.Pow(10, float64(d)) + for _, u := range []struct { + scale float64 + suffix string + }{{1e12, "T"}, {1e9, "B"}, {1e6, "M"}, {1e3, "K"}} { + if math.Round(abs/u.scale*pow)/pow >= 1 { + return fixed(f/u.scale, d, false) + u.suffix + } + } + return fixed(f, d, false) +} + +func isTemporal(dataType string) bool { + u := strings.ToUpper(dataType) + return strings.Contains(u, "DATE") || strings.Contains(u, "TIME") +} + +func parseISO(s string) (time.Time, bool) { + for _, layout := range []string{"2006-01-02 15:04:05.000", "2006-01-02 15:04:05", "2006-01-02T15:04:05Z07:00", "2006-01-02T15:04:05", "2006-01-02"} { + if t, err := time.Parse(layout, s); err == nil { + return t, true + } + } + return time.Time{}, false +} + +// strftime handles the common directives; unknown ones are left in place. +func strftime(t time.Time, pattern string) string { + var b strings.Builder + for i := 0; i < len(pattern); i++ { + if pattern[i] != '%' || i+1 >= len(pattern) { + b.WriteByte(pattern[i]) + continue + } + i++ + switch pattern[i] { + case 'Y': + b.WriteString(t.Format("2006")) + case 'y': + b.WriteString(t.Format("06")) + case 'm': + b.WriteString(t.Format("01")) + case 'd': + b.WriteString(t.Format("02")) + case 'e': + b.WriteString(t.Format("_2")) + case 'b': + b.WriteString(t.Format("Jan")) + case 'B': + b.WriteString(t.Format("January")) + case 'a': + b.WriteString(t.Format("Mon")) + case 'A': + b.WriteString(t.Format("Monday")) + case 'H': + b.WriteString(t.Format("15")) + case 'I': + b.WriteString(t.Format("03")) + case 'M': + b.WriteString(t.Format("04")) + case 'S': + b.WriteString(t.Format("05")) + case 'p': + b.WriteString(t.Format("PM")) + case 'j': + fmt.Fprintf(&b, "%03d", t.YearDay()) + case '%': + b.WriteByte('%') + default: + b.WriteByte('%') + b.WriteByte(pattern[i]) + } + } + return b.String() +} + +func formatTime(t time.Time, dataType string) string { + dateOnly := strings.EqualFold(dataType, "DATE") || + (t.Hour() == 0 && t.Minute() == 0 && t.Second() == 0 && t.Nanosecond() == 0) + if dateOnly { + return t.Format("Jan 2, 2006") + } + return t.Format("Jan 2, 2006 15:04") +} + +func asFloat(v any) (float64, bool) { + switch x := v.(type) { + case int64: + return float64(x), true + case float64: + return x, true + case int: + return float64(x), true + case result.Decimal: + return x.Float64(), true + } + return 0, false +} diff --git a/internal/output/format_test.go b/internal/output/format_test.go new file mode 100644 index 0000000..e54acb6 --- /dev/null +++ b/internal/output/format_test.go @@ -0,0 +1,247 @@ +package output + +import ( + "math" + "math/big" + "testing" + "time" + + "github.com/exploreomni/omni-cli/internal/result" +) + +func TestFormatValue_ExactIntegers(t *testing.T) { + for _, tc := range []struct { + value int64 + format, want string + }{ + {9007199254740993, "id", "9007199254740993"}, + {math.MaxInt64, "id", "9223372036854775807"}, + {math.MinInt64, "id", "-9223372036854775808"}, + {9007199254740993, "", "9,007,199,254,740,993"}, + {9007199254740993, "NUMBER_0", "9,007,199,254,740,993"}, + {9007199254740993, "number", "9,007,199,254,740,993.00"}, + {9007199254740993, "currency_2", "$9,007,199,254,740,993.00"}, + {math.MinInt64, "accounting_0", "$(9,223,372,036,854,775,808)"}, + {-9007199254740993, "financial_0", "(9,007,199,254,740,993)"}, + {9007199254740993, "#,##0.0#", "9,007,199,254,740,993.0"}, + {math.MinInt64, "#,##0;(#,##0)", "(9,223,372,036,854,775,808)"}, + {-9007199254740993, "0", "-9007199254740993"}, + {12345, "big_1", "12.3K"}, + {12, "percent_0", "1,200%"}, + {1234, "0.0,", "1.2"}, + {-1234, "0.0,;(0.0,)", "(1.2)"}, + {1234, "0.00E+00", "1.23E+03"}, + } { + if got := FormatValue(tc.value, result.Column{Format: tc.format}); got != tc.want { + t.Errorf("FormatValue(%d, %q) = %q, want %q", tc.value, tc.format, got, tc.want) + } + } +} + +func dec(t *testing.T, s string, scale int32) result.Decimal { + t.Helper() + coef, ok := new(big.Int).SetString(s, 10) + if !ok { + t.Fatalf("bad coefficient %q", s) + } + return result.Decimal{Coef: coef, Scale: scale} +} + +func TestFormatValue_ExactDecimals(t *testing.T) { + for _, tc := range []struct { + value result.Decimal + format, want string + }{ + {dec(t, "12345678901234567891", 2), "", "123,456,789,012,345,678.91"}, + {dec(t, "1250", 2), "", "12.5"}, + {dec(t, "-100", 2), "", "-1"}, + {dec(t, "5", 3), "", "0.005"}, + {dec(t, "18446744073709551615", 0), "id", "18446744073709551615"}, + {dec(t, "18446744073709551615", 0), "", "18,446,744,073,709,551,615"}, + {dec(t, "12345678901234567891", 2), "number_2", "123,456,789,012,345,678.91"}, + {dec(t, "12345678901234567895", 3), "number_2", "12,345,678,901,234,567.90"}, + {dec(t, "999995", 4), "number_1", "100.0"}, + {dec(t, "5", 1), "id", "1"}, + {dec(t, "-4", 3), "number_2", "0.00"}, + {dec(t, "-12345678901234567891", 2), "currency_2", "-$123,456,789,012,345,678.91"}, + {dec(t, "-12345678901234567891", 2), "financial_0", "(123,456,789,012,345,679)"}, + {dec(t, "12345678901234567891", 2), "#,##0.0#", "123,456,789,012,345,678.91"}, + {dec(t, "1250", 2), "0.0#", "12.5"}, + {dec(t, "1250", 2), "percent_0", "1,250%"}, + } { + if got := FormatValue(tc.value, result.Column{Format: tc.format}); got != tc.want { + t.Errorf("FormatValue(%s, %q) = %q, want %q", tc.value, tc.format, got, tc.want) + } + } +} + +func TestFormatValue_ZeroPaddedPatterns(t *testing.T) { + for _, tc := range []struct { + value any + format, want string + }{ + {int64(12), "00000", "00012"}, + {int64(-12), "00000", "-00012"}, + {int64(123456), "00000", "123456"}, + {12.6, "00000", "00013"}, + {3.14159, "000.00", "003.14"}, + {int64(1234), "#,##0000", "1,234"}, + {int64(12), "#,000,000", "000,012"}, + {dec(t, "125", 1), "0000.0", "0012.5"}, + {0.5, "#.##", "0.5"}, + } { + if got := FormatValue(tc.value, result.Column{Format: tc.format}); got != tc.want { + t.Errorf("FormatValue(%v, %q) = %q, want %q", tc.value, tc.format, got, tc.want) + } + } +} + +func TestFormatValue_SanitizesModelFormats(t *testing.T) { + for _, tc := range []struct { + value any + format, want string + }{ + {int64(1), "0\"\x1b]0;changed\a\n\t\"", "1]0;changed "}, + {1.5, "0.0\"\x1b]0;changed\a\"", "1.5]0;changed"}, + {time.Date(2026, 9, 14, 0, 0, 0, 0, time.UTC), "%Y\x1b[2J\n", "2026[2J "}, + {"2026-09-14", "%Y\x1b[2J\n", "2026[2J "}, + } { + if got := FormatValue(tc.value, result.Column{Format: tc.format, DataType: "DATE"}); got != tc.want { + t.Errorf("FormatValue(%v, %q) = %q, want %q", tc.value, tc.format, got, tc.want) + } + } +} + +func TestFormatValue_ModelFormats(t *testing.T) { + tests := []struct { + format string + in any + want string + }{ + // Numeric names, with the documented default decimals. + {"number", 1234.5, "1,234.50"}, + {"NUMBER_0", int64(12526), "12,526"}, + {"NUMBER_0", 838.7, "839"}, + {"NUMBER_2", 1602513.8052, "1,602,513.81"}, + {"number_1", 0.26, "0.3"}, + {"percent", 0.244, "24.4%"}, + {"PERCENT_0", 0.449, "45%"}, + {"percent_2", 0.3992495609133003, "39.92%"}, + {"PERCENT_1", -0.0512, "-5.1%"}, + {"id", int64(123450), "123450"}, + {"ID", 9007199254740992.0, "9007199254740992"}, + {"big", 5.6e6, "5.60M"}, + {"big_1", 1284220.5, "1.3M"}, + {"big_2", 9120.0, "9.12K"}, + {"big_0", 3.4e9, "3B"}, + {"big_1", 42.0, "42.0"}, + {"billions", 1.2e9, "1.20B"}, + {"millions", 5.6e6, "5.6M"}, + {"thousands", 8900.0, "8.90K"}, + // Currency families: category, optional currency prefix, decimals. + {"currency", 1234.5, "$1,234.50"}, + {"currency_0", -987654.0, "-$987,654"}, + {"usdcurrency_2", int64(5), "$5.00"}, + {"USDCURRENCY_0", 150000.0, "$150,000"}, + {"gbpcurrency_2", -1234.5, "-£1,234.50"}, + {"eurcurrency_2", 1234.5, "€1,234.50"}, + {"jpycurrency_0", 1234.0, "¥1,234"}, + {"brlcurrency_2", 10.0, "R$10.00"}, + {"accounting", -1234.5, "$(1,234.50)"}, + {"ACCOUNTING_0", 150000.0, "$150,000"}, + {"usdaccounting_2", -1234.5, "$(1,234.50)"}, + {"jpyaccounting_0", -1234.0, "¥(1,234)"}, + {"audfinancial_2", -1234.5, "(1,234.50)"}, + {"financial_0", 1234.6, "1,235"}, + {"bigusdcurrency_2", 5.6e6, "$5.60M"}, + {"bigeurcurrency_2", -5.6e6, "-€5.60M"}, + {"bigaccounting_1", -2.5e3, "$(2.5K)"}, + {"bigfinancial_0", 3.4e9, "3B"}, + // Excel-style patterns, from the docs and from production models. + {"#,##0", 1234.0, "1,234"}, + {"#,##0.00", 1234.56, "1,234.56"}, + {"0%", 0.75, "75%"}, + {"0.00%", 0.75, "75.00%"}, + {"0.0%", 0.3992, "39.9%"}, + {`0.00"%"`, 63.4567, "63.46%"}, + {`0.00"%"`, -2.0, "-2.00%"}, + {"0.00E+00", 1234.0, "1.23E+03"}, + {`#,##0 "units"`, 1234.0, "1,234 units"}, + {`#,##0.00 "kg"`, 1234.5, "1,234.50 kg"}, + {"#,##0.0,", 1234.0, "1.2"}, + {`#,##0.0,,"M"`, 1234567.0, "1.2M"}, + {"$#,##0.00", 1234.5, "$1,234.50"}, + {"$#,##0.00", -1234.5, "-$1,234.50"}, + {`\$0.0`, 2.26, "$2.3"}, + // £ and ¥ are two bytes, € three: each is consumed whole, wherever it sits. + {"£0.00", 1234.5, "£1234.50"}, + {"£#,##0", 1234.0, "£1,234"}, + {"¥#,##0", 1234.0, "¥1,234"}, + {"0.00£", 1234.5, "1234.50£"}, + {"#,##0€", 1234.0, "1,234€"}, + // # decimals show only when they aren't trailing zeros. + {"#,##0.##", 1234.567, "1,234.57"}, + {"#,##0.##", 1234.5, "1,234.5"}, + {"#,##0.##", 1234.0, "1,234"}, + {"0.0#", 2.0, "2.0"}, + // The unit is picked after rounding. + {"big", 999999.9, "1.00M"}, + {"big_0", 999.6, "1K"}, + // Parenthesised negatives: the standard accounting shape. + {"#,##0;(#,##0)", -1234.0, "(1,234)"}, + {"#,##0;(#,##0)", 1234.0, "1,234"}, + {"$#,##0.00;($#,##0.00)", -1234.5, "($1,234.50)"}, + {`"🚀 "0.0;"📉 "-0.0;0`, 1.5, "🚀 1.5"}, + {`"🚀 "0.0;"📉 "-0.0;0`, -1.5, "📉 -1.5"}, + {`"🚀 "0.0;"📉 "-0.0;0`, 0.0, "0"}, + // Beyond a client's reach: field references need other columns. + {`#,##0.00 "{{orders.currency_symbol.value}}"`, 1234.5, "1234.5"}, + // Unknown names and patterns fall back to the readable default. + {"[h]:mm:ss", 3661.0, "3661"}, + {"DURATION", 3661.0, "3661"}, + {"", 1602513.8052352013, "1,602,513.8052352013"}, + {"", int64(2026), "2026"}, + } + for _, tc := range tests { + got := FormatValue(tc.in, result.Column{Format: tc.format}) + if got != tc.want { + t.Errorf("FormatValue(%v, %q) = %q, want %q", tc.in, tc.format, got, tc.want) + } + } +} + +func TestFormatValue_NonNumbers(t *testing.T) { + d := time.Date(2026, 8, 10, 0, 0, 0, 0, time.UTC) + ts := time.Date(2026, 8, 10, 14, 30, 0, 0, time.UTC) + tests := []struct { + in any + col result.Column + want string + }{ + {nil, result.Column{}, "-"}, + {"", result.Column{}, "-"}, + {"Ireland", result.Column{}, "Ireland"}, + {true, result.Column{}, "true"}, + {d, result.Column{DataType: "DATE"}, "Aug 10, 2026"}, + {d, result.Column{DataType: "TIMESTAMP"}, "Aug 10, 2026"}, + {ts, result.Column{DataType: "TIMESTAMP"}, "Aug 10, 2026 14:30"}, + {ts, result.Column{DataType: "DATETIME"}, "Aug 10, 2026 14:30"}, + // A model date format applies to decoded timestamps too. + {ts, result.Column{DataType: "TIMESTAMP", Format: "%Y-%m"}, "2026-08"}, + {d, result.Column{DataType: "DATE", Format: "%d/%m/%Y"}, "10/08/2026"}, + // Temporal dimensions arrive formatted by the model, in its reporting + // timezone, and are shown as sent. + {"2026-08-05 11:31:00.000", result.Column{DataType: "TIMESTAMP"}, "2026-08-05 11:31:00.000"}, + {"2026-08-10", result.Column{DataType: "DATE"}, "2026-08-10"}, + // A strftime pattern on a temporal dimension is applied to the ISO + // text the API sends, the way the model asks. + {"2026-09-12", result.Column{DataType: "TIMESTAMP", Format: "%d-%m-%Y"}, "12-09-2026"}, + {"2026-09-12 14:05:09.000", result.Column{DataType: "TIMESTAMP", Format: "%b %d, %Y %H:%M"}, "Sep 12, 2026 14:05"}, + {"not a date", result.Column{DataType: "TIMESTAMP", Format: "%d-%m-%Y"}, "not a date"}, + } + for _, tc := range tests { + if got := FormatValue(tc.in, tc.col); got != tc.want { + t.Errorf("FormatValue(%v) = %q, want %q", tc.in, got, tc.want) + } + } +} diff --git a/internal/output/human.go b/internal/output/human.go index c04832f..5b81cda 100644 --- a/internal/output/human.go +++ b/internal/output/human.go @@ -5,8 +5,10 @@ import ( "encoding/json" "fmt" "io" + "math" "os" "sort" + "strconv" "strings" "time" "unicode" @@ -45,7 +47,7 @@ func HumanBytes(w io.Writer, data []byte) error { fmt.Fprintln(w) return werr } - renderValue(w, v) + renderValue(w, sanitizeJSON(v)) return nil } @@ -56,6 +58,7 @@ func HumanError(statusCode int, detail string) { // HumanErrorTo prints a plain-text error message to w. func HumanErrorTo(w io.Writer, statusCode int, detail string) { + detail = sanitize(detail) if detail == "" { detail = fmt.Sprintf("HTTP %d", statusCode) } @@ -237,28 +240,25 @@ func renderTable(w io.Writer, rows []any) { } t := table.New(). - Border(lipgloss.NormalBorder()). - BorderStyle(lipgloss.NewStyle().Foreground(lipgloss.Color("240"))). + Border(lipgloss.RoundedBorder()). + BorderStyle(styleBorder). Headers(headers...). StyleFunc(styleFor(columns)) for _, rec := range records { row := make([]string, len(columns)) for i, c := range columns { - row[i] = truncate(formatScalar(rec[c]), 60) + row[i] = truncateCells(formatField(c, rec[c]), 60) } t.Row(row...) } fmt.Fprintln(w, t.Render()) } -// styleFor returns a StyleFunc that dims identifier columns and greys out -// timestamps, keeping names and other scalars at default foreground. +// styleFor returns a StyleFunc that mutes identifier and timestamp columns, +// keeping names and other scalars at default foreground. func styleFor(columns []string) func(row, col int) lipgloss.Style { - dim := lipgloss.NewStyle().Padding(0, 1).Foreground(lipgloss.Color("8")) - grey := lipgloss.NewStyle().Padding(0, 1).Foreground(lipgloss.Color("244")) - header := lipgloss.NewStyle().Padding(0, 1).Bold(true) - base := lipgloss.NewStyle().Padding(0, 1) + dim, grey, header, base := styleMuted, styleMuted, styleHeader, styleCell return func(row, col int) lipgloss.Style { if row == table.HeaderRow { @@ -310,7 +310,7 @@ func renderKeyValue(w io.Writer, obj map[string]any) { fmt.Fprintf(w, "%-*s %s\n", maxKey+1, labels[k]+":", summarizeComplex(v)) continue } - fmt.Fprintf(w, "%-*s %s\n", maxKey+1, labels[k]+":", formatScalar(v)) + fmt.Fprintf(w, "%-*s %s\n", maxKey+1, labels[k]+":", formatField(k, v)) } } @@ -406,6 +406,26 @@ func summarizeComplex(v any) string { return "" } +// formatField renders a value under the field name it arrived with. An +// identifier is a value someone copies back into a command, so it keeps its +// digits ungrouped; everything else reads better with separators. +func formatField(key string, v any) string { + if f, ok := v.(float64); ok && identifierKey(key) { + return formatNumberPlain(f) + } + return formatScalar(v) +} + +// identifierKey reports whether a field name reads as an identifier or a port +// rather than a magnitude. +func identifierKey(k string) bool { + switch k { + case "id", "port", "version": + return true + } + return strings.HasSuffix(k, "Id") || strings.HasSuffix(k, "ID") +} + func formatScalar(v any) string { switch x := v.(type) { case nil: @@ -424,11 +444,7 @@ func formatScalar(v any) string { } return "false" case float64: - // JSON numbers decode as float64; print without trailing .0 when integral. - if x == float64(int64(x)) { - return fmt.Sprintf("%d", int64(x)) - } - return fmt.Sprintf("%g", x) + return formatNumber(x) case json.Number: return x.String() } @@ -436,6 +452,88 @@ func formatScalar(v any) string { return string(b) } +// formatNumber renders a JSON number for reading: no exponent, no invented +// precision, and thousands separators once the digits outrun a glance. +// Separators start at five digits so a year stays 2026 rather than 2,026. +func formatNumber(f float64) string { + return formatNumberGrouped(f, 5) +} + +// formatNumberPlain is formatNumber without separators, for fields named as +// identifiers: a grouped id is a value someone copies back wrong. +func formatNumberPlain(f float64) string { + return formatNumberGrouped(f, math.MaxInt) +} + +// formatNumberGrouped groups thousands once the integer part has minDigits digits. +func formatNumberGrouped(f float64, minDigits int) string { + if math.IsNaN(f) || math.IsInf(f, 0) { + return fmt.Sprintf("%g", f) + } + abs := math.Abs(f) + integral := f == math.Trunc(f) + // Outside these ranges the plain form is longer than it is useful. Whole + // numbers get far more room: a 16-digit warehouse ID is a value someone + // reads, not a magnitude they skim. + switch { + case abs == 0: + case integral && abs >= 1e18: + return fmt.Sprintf("%g", f) + case !integral && (abs >= 1e15 || abs < 1e-6): + return fmt.Sprintf("%g", f) + } + + // Shortest form that round-trips: a coordinate keeps its digits. Model + // formats, not this, decide decimals for query results. + s := strconv.FormatFloat(f, 'f', -1, 64) + + sign := "" + if strings.HasPrefix(s, "-") { + sign, s = "-", s[1:] + } + intPart, frac := s, "" + if i := strings.IndexByte(s, '.'); i >= 0 { + intPart, frac = s[:i], s[i:] + } + if len(intPart) < minDigits { + return sign + intPart + frac + } + + var b strings.Builder + for i := range intPart { + if i > 0 && (len(intPart)-i)%3 == 0 { + b.WriteByte(',') + } + b.WriteByte(intPart[i]) + } + return sign + b.String() + frac +} + +// truncateCells shortens s to max terminal cells. Runes aren't cells: a CJK +// label of 28 runes occupies 56 columns, which would blow the layout its +// width was budgeted for. +func truncateCells(s string, max int) string { + switch { + case lipgloss.Width(s) <= max: + return s + case max <= 0: + return "" + case max == 1: + return "…" + } + var b strings.Builder + used := 0 + for _, r := range s { + w := lipgloss.Width(string(r)) + if used+w > max-1 { + break + } + b.WriteRune(r) + used += w + } + return b.String() + "…" +} + func parseTime(s string) (time.Time, bool) { // Common Omni API timestamp shapes. layouts := []string{time.RFC3339Nano, time.RFC3339, "2006-01-02T15:04:05"} @@ -468,13 +566,6 @@ func relativeTime(t time.Time) string { } } -func truncate(s string, max int) string { - if max <= 1 || len(s) <= max { - return s - } - return s[:max-1] + "…" -} - // humanizeKey converts API field names like "modelKind", "MODEL_KIND", or // "created_at" into readable labels like "Model Kind" / "Created At". // camelCase splits on case transitions; snake_case / kebab-case become spaces. diff --git a/internal/output/human_test.go b/internal/output/human_test.go index 0625ae2..c481067 100644 --- a/internal/output/human_test.go +++ b/internal/output/human_test.go @@ -4,6 +4,7 @@ import ( "bytes" "strings" "testing" + "unicode/utf8" ) // List with `records` + `pageInfo` (the models/dashboards shape). @@ -217,3 +218,117 @@ func TestHumanErrorTo(t *testing.T) { t.Errorf("expected status code, got: %q", out) } } + +func TestFormatNumber(t *testing.T) { + tests := map[float64]string{ + 0: "0", + 999: "999", + 1000: "1000", + // Separators start at five digits, so a year is left alone. + 2026: "2026", + 10000: "10,000", + -1234567: "-1,234,567", + // No exponent, and every digit kept: model formats decide decimals + // for query results; the generic table shows what the API sent. + 1284220.5: "1,284,220.5", + 1602513.8052352013: "1,602,513.8052352013", + 1.005: "1.005", + 37.774929: "37.774929", + // Below 1 the digits are the whole story, so they all survive. + 0.123456: "0.123456", + -0.25: "-0.25", + 1e9: "1,000,000,000", + // Whole numbers stay readable well past the float64 integer range; + // a warehouse ID is read, not skimmed. + 9007199254740992: "9,007,199,254,740,992", + // Beyond that, fall back to the compact form. + 1e18: "1e+18", + // A tiny fraction is a magnitude, not a number anyone reads digit by + // digit, so it keeps the compact form. + 1e-9: "1e-09", + } + for in, want := range tests { + if got := formatNumber(in); got != want { + t.Errorf("formatNumber(%v) = %q, want %q", in, got, want) + } + } +} + +// Large floats used to reach the table as scientific notation. +func TestHumanBytes_LargeFloatInTable(t *testing.T) { + var buf bytes.Buffer + body := []byte(`[{"region":"east","revenue":1284220.5}]`) + if err := HumanBytes(&buf, body); err != nil { + t.Fatalf("HumanBytes: %v", err) + } + out := buf.String() + if !strings.Contains(out, "1,284,220.5") { + t.Errorf("expected a readable number, got:\n%s", out) + } + if strings.Contains(out, "e+") { + t.Errorf("expected no scientific notation, got:\n%s", out) + } +} + +// An id is a value someone copies back into a command, so it keeps its +// digits; the measure beside it still reads as a magnitude. +func TestHumanBytes_IdentifiersAreNotGrouped(t *testing.T) { + var buf bytes.Buffer + body := []byte(`[{"id":123456,"connectionId":987654,"revenue":1284220.5}]`) + if err := HumanBytes(&buf, body); err != nil { + t.Fatalf("HumanBytes: %v", err) + } + out := buf.String() + for _, want := range []string{"123456", "987654", "1,284,220.5"} { + if !strings.Contains(out, want) { + t.Errorf("expected %q in:\n%s", want, out) + } + } +} + +func TestTruncate_DoesNotSplitRunes(t *testing.T) { + s := "متجر إلكتروني - لوحة المبيعات" + got := truncateCells(s, 10) + if !utf8.ValidString(got) { + t.Errorf("truncate produced invalid UTF-8: %q", got) + } + if n := utf8.RuneCountInString(got); n != 10 { + t.Errorf("expected 10 runes, got %d (%q)", n, got) + } + if truncateCells("short", 10) != "short" { + t.Error("a string under the limit should pass through") + } + if got := truncateCells("Stage", 1); got != "…" { + t.Errorf("truncateCells(_, 1) = %q, want an ellipsis", got) + } + if got := truncateCells("Stage", 0); got != "" { + t.Errorf("truncateCells(_, 0) = %q, want nothing", got) + } + if got := truncateCells("#", 1); got != "#" { + t.Errorf("a one-cell string fits one cell, got %q", got) + } +} + +// Values are data: an escape sequence in one must not reach the terminal. +func TestHumanBytes_StripsControlCharacters(t *testing.T) { + var buf bytes.Buffer + body := `{"records":[{"id":"1","name":"evil\u001b]52;c;aGk=\u0007name\u001b[2J"}]}` + if err := HumanBytes(&buf, []byte(body)); err != nil { + t.Fatal(err) + } + if out := buf.String(); strings.ContainsAny(out, "\x1b\x07") || !strings.Contains(out, "evil]52;c;aGk=name[2J") { + t.Errorf("control characters should be dropped, text kept:\n%q", out) + } +} + +func TestSanitizeJSON_KeyCollisionKeepsBothFields(t *testing.T) { + v := sanitizeJSON(map[string]any{"ab": "clean", "a\x00b": "dirty"}).(map[string]any) + if len(v) != 2 || v["ab"] != "clean" || v["ab (2)"] != "dirty" { + t.Errorf("expected both fields kept, got %#v", v) + } + for k := range v { + if strings.ContainsRune(k, 0) { + t.Errorf("key %q still has a control character", k) + } + } +} diff --git a/internal/output/result.go b/internal/output/result.go new file mode 100644 index 0000000..83f2b5f --- /dev/null +++ b/internal/output/result.go @@ -0,0 +1,399 @@ +package output + +import ( + "fmt" + "io" + "strings" + "unicode" + + "github.com/charmbracelet/lipgloss" + "github.com/charmbracelet/lipgloss/table" + "github.com/charmbracelet/x/ansi" + "github.com/exploreomni/omni-cli/internal/result" +) + +// ResultTable renders a decoded query result with the model's labels and +// formats, pivoted when the query pivots. +func ResultTable(w io.Writer, set *result.Set) { + if len(set.Rows) == 0 { + fmt.Fprintln(w, "No results.") + return + } + if p := set.Pivot(); p != nil { + pivotTable(w, set, p) + return + } + headers := make([]string, len(set.Columns)) + for i, c := range set.Columns { + headers[i] = singleLine(c.Label) + } + + t := resultTable(headers, func(col int) bool { + return col < len(set.Columns) && !set.Columns[col].IsDimension + }) + for _, r := range set.Rows { + cells := make([]string, len(set.Columns)) + for i, c := range set.Columns { + cells[i] = truncateCells(FormatValue(r[i], c), 60) + } + t.Row(cells...) + } + fmt.Fprintln(w, t.Render()) +} + +// pivotTable lays a pivot out as the Omni app does: the row dimensions on +// the left, then a column per pivot value and measure, headed by the pivot +// value over the measure's label. +func pivotTable(w io.Writer, set *result.Set, p *result.Pivoted) { + group := pivotLabel(set, p) + var headers []string + for i, c := range p.RowDims { + top := "" + if i == len(p.RowDims)-1 { + top = group + } + headers = append(headers, top+"\n"+singleLine(set.Columns[c].Label)) + } + for _, key := range p.Keys { + for _, m := range p.Measures { + headers = append(headers, pivotKey(set, p, key)+"\n"+singleLine(set.Columns[m].Label)) + } + } + + // Table headers are one line, so the two-line header is the first row, + // and of the row borders only the one beneath it is kept. + t := table.New(). + Border(lipgloss.RoundedBorder()). + BorderStyle(styleBorder). + BorderRow(true). + StyleFunc(func(row, col int) lipgloss.Style { + if row == 0 { + return styleHeader + } + if col >= len(p.RowDims) { + return styleNum + } + return styleCell + }). + Row(headers...) + for _, r := range p.Rows { + var cells []string + for i, c := range p.RowDims { + cells = append(cells, truncateCells(FormatValue(r.Dims[i], set.Columns[c]), 60)) + } + for k := range p.Keys { + for j, m := range p.Measures { + var v any + if r.Cells[k] != nil { + v = r.Cells[k][j] + } + cells = append(cells, FormatValue(v, set.Columns[m])) + } + } + t.Row(cells...) + } + lines := strings.Split(t.Render(), "\n") + kept := lines[:0] + separators := 0 + for i, line := range lines { + if i > 0 && i < len(lines)-1 && strings.HasPrefix(ansi.Strip(line), "├") { + if separators++; separators > 1 { + continue + } + } + kept = append(kept, line) + } + fmt.Fprintln(w, strings.Join(kept, "\n")) + if p.Omitted > 0 { + fmt.Fprintln(w, styleDim.Render(fmt.Sprintf("… and %d more pivot column%s past the query's column limit", p.Omitted, plural(p.Omitted)))) + } +} + +func resultTable(headers []string, numeric func(col int) bool) *table.Table { + return table.New(). + Border(lipgloss.RoundedBorder()). + BorderStyle(styleBorder). + Headers(headers...). + StyleFunc(func(row, col int) lipgloss.Style { + if row == table.HeaderRow { + return styleHeader + } + if col >= 0 && numeric(col) { + return styleNum + } + return styleCell + }) +} + +// pivotLabel names what the pivot columns are values of, e.g. "Stage". +func pivotLabel(set *result.Set, p *result.Pivoted) string { + labels := make([]string, len(p.PivotDims)) + for i, c := range p.PivotDims { + labels[i] = singleLine(set.Columns[c].Label) + } + return strings.Join(labels, " · ") +} + +// pivotKey renders one pivot column's values, e.g. "Closed Won". +func pivotKey(set *result.Set, p *result.Pivoted, key []any) string { + parts := make([]string, len(key)) + for i, v := range key { + parts[i] = FormatValue(v, set.Columns[p.PivotDims[i]]) + } + return strings.Join(parts, " · ") +} + +// ResultChart draws the result as the Omni app's bar table: each dimension +// a column, each measure a column of bars on its own scale. A pivoted query +// spreads its measures across the pivot values, which share that scale. +// --chart-value narrows the bars to the measures it names. +func ResultChart(w io.Writer, set *result.Set, opts ChartOptions) error { + if len(set.Rows) == 0 { + fmt.Fprintln(w, "No results.") + return nil + } + var g *grid + var err error + if p := set.Pivot(); p != nil { + g, err = pivotGrid(set, p, opts) + } else { + g, err = flatGrid(set, opts) + } + if err != nil { + return err + } + renderGrid(w, g, opts) + return nil +} + +func flatGrid(set *result.Set, opts ChartOptions) (*grid, error) { + values, err := pickValues(set, opts.Values) + if err != nil { + return nil, err + } + var labels []int + for i, c := range set.Columns { + if c.IsDimension && indexOfInt(values, i) < 0 { + labels = append(labels, i) + } + } + + rows, omitted := capRows(len(set.Rows), opts) + g := &grid{omittedRows: omitted} + for _, c := range labels { + g.labelHeaders = append(g.labelHeaders, singleLine(set.Columns[c].Label)) + } + for s, v := range values { + g.cols = append(g.cols, gridCol{header: singleLine(set.Columns[v].Label), scale: s}) + } + for i, r := range set.Rows[:rows] { + var ls []string + for _, c := range labels { + ls = append(ls, FormatValue(r[c], set.Columns[c])) + } + if len(labels) == 0 { + ls = []string{fmt.Sprintf("%d", i+1)} + } + g.labels = append(g.labels, ls) + for s, v := range values { + g.cols[s].items = append(g.cols[s].items, chartItem(r[v], set.Columns[v])) + } + } + if len(labels) == 0 { + g.labelHeaders = []string{"#"} + } + return g, nil +} + +func pivotGrid(set *result.Set, p *result.Pivoted, opts ChartOptions) (*grid, error) { + measures := p.Measures + if len(opts.Values) > 0 { + measures = nil + for _, want := range opts.Values { + i, ok := matchColumn(set.Columns, want) + if !ok { + return nil, fmt.Errorf("--chart-value %q is not a column (have: %s)", want, columnNames(set)) + } + if indexOfInt(p.Measures, i) < 0 || !numericColumn(set, i) { + return nil, fmt.Errorf("--chart-value %q is not a measure this pivot spreads across its columns", set.Columns[i].Label) + } + if indexOfInt(measures, i) < 0 { + measures = append(measures, i) + } + } + } + var numeric []int + for _, m := range measures { + if numericColumn(set, m) { + numeric = append(numeric, m) + } + } + if len(numeric) == 0 { + return nil, fmt.Errorf("--chart found nothing numeric to plot (have: %s)", columnNames(set)) + } + + labels := p.RowDims + rows, omitted := capRows(len(p.Rows), opts) + g := &grid{groupLabel: pivotLabel(set, p), omittedRows: omitted, omittedCols: p.Omitted * len(numeric)} + for _, c := range labels { + g.labelHeaders = append(g.labelHeaders, singleLine(set.Columns[c].Label)) + } + if len(labels) == 0 { + g.labelHeaders = []string{"#"} + } + for k, key := range p.Keys { + for s, m := range numeric { + header := singleLine(set.Columns[m].Label) + // One measure needs naming once; with several, each column says which. + if len(numeric) == 1 && k > 0 { + header = "" + } + g.cols = append(g.cols, gridCol{group: pivotKey(set, p, key), header: header, scale: s}) + } + } + for i, r := range p.Rows[:rows] { + var ls []string + for _, c := range labels { + ls = append(ls, FormatValue(r.Dims[indexOfInt(p.RowDims, c)], set.Columns[c])) + } + if len(labels) == 0 { + ls = []string{fmt.Sprintf("%d", i+1)} + } + g.labels = append(g.labels, ls) + col := 0 + for k := range p.Keys { + for _, m := range numeric { + var v any + if r.Cells[k] != nil { + v = r.Cells[k][indexOfInt(p.Measures, m)] + } + g.cols[col].items = append(g.cols[col].items, chartItem(v, set.Columns[m])) + col++ + } + } + } + return g, nil +} + +func chartItem(v any, c result.Column) chartRow { + var it chartRow + if f, ok := asFloat(v); ok { + it.value, it.present = f, true + it.text = FormatValue(v, c) + } + return it +} + +func capRows(n int, opts ChartOptions) (rows, omitted int) { + maxRows := opts.MaxRows + if maxRows <= 0 { + maxRows = DefaultChartRows + } + if n > maxRows { + return maxRows, n - maxRows + } + return n, 0 +} + +// pickValues chooses the columns to draw bars for: the ones --chart-value +// names, in that order, else every measure, else the first numeric column. +func pickValues(set *result.Set, wants []string) ([]int, error) { + if len(wants) > 0 { + var picked []int + for _, want := range wants { + i, ok := matchColumn(set.Columns, want) + if !ok { + return nil, fmt.Errorf("--chart-value %q is not a column (have: %s)", want, columnNames(set)) + } + if !numericColumn(set, i) { + return nil, fmt.Errorf("--chart-value %q holds no numbers", set.Columns[i].Label) + } + if indexOfInt(picked, i) < 0 { + picked = append(picked, i) + } + } + return picked, nil + } + var measures []int + for i, c := range set.Columns { + if !c.IsDimension && numericColumn(set, i) { + measures = append(measures, i) + } + } + if len(measures) > 0 { + return measures, nil + } + for i := range set.Columns { + if numericColumn(set, i) { + return []int{i}, nil + } + } + return nil, fmt.Errorf("--chart found nothing numeric to plot (have: %s)", columnNames(set)) +} + +func indexOfInt(s []int, v int) int { + for i, x := range s { + if x == v { + return i + } + } + return -1 +} + +func numericColumn(set *result.Set, i int) bool { + for _, r := range set.Rows { + if r[i] == nil { + continue + } + if _, ok := asFloat(r[i]); ok { + return true + } + return false + } + return false +} + +func columnNames(set *result.Set) string { + names := make([]string, len(set.Columns)) + for i, c := range set.Columns { + names[i] = c.Name + } + return strings.Join(names, ", ") +} + +// matchColumn finds a column by field name or label: exact, then normalized +// ("engaged_sessions_percent" ~ "Engaged Sessions %"), then without a view +// prefix on either side ("count" ~ "deals.count" labelled "Deals Count"). +func matchColumn(columns []result.Column, want string) (int, bool) { + for i, c := range columns { + if c.Name == want || c.Label == want { + return i, true + } + } + norm := normalizeColumn(want) + for i, c := range columns { + if normalizeColumn(c.Name) == norm || normalizeColumn(c.Label) == norm { + return i, true + } + } + for i, c := range columns { + if j := strings.LastIndexByte(c.Name, '.'); j >= 0 && normalizeColumn(c.Name[j+1:]) == norm { + return i, true + } + } + if i := strings.LastIndexByte(want, '.'); i >= 0 { + return matchColumn(columns, want[i+1:]) + } + return 0, false +} + +func normalizeColumn(s string) string { + s = strings.ToLower(strings.ReplaceAll(s, "%", "percent")) + var b strings.Builder + for _, r := range s { + if unicode.IsLetter(r) || unicode.IsDigit(r) { + b.WriteRune(r) + } + } + return b.String() +} diff --git a/internal/output/sanitize.go b/internal/output/sanitize.go new file mode 100644 index 0000000..18e315d --- /dev/null +++ b/internal/output/sanitize.go @@ -0,0 +1,74 @@ +package output + +import ( + "fmt" + "sort" + "strings" + "unicode" +) + +// Text from the API and the warehouse is data, not terminal instructions: an +// ESC in a value would otherwise let it retitle the window or write the +// clipboard (OSC 52) when rendered. JSON and passed-through payloads are +// written as they arrived; only human rendering goes through these. + +// sanitize drops control characters, keeping newlines and tabs. +func sanitize(s string) string { + if strings.IndexFunc(s, isUnsafe) < 0 { + return s + } + return strings.Map(func(r rune) rune { + if isUnsafe(r) { + return -1 + } + return r + }, s) +} + +// singleLine sanitizes text bound for one cell of a layout, where a newline +// would break the rows. +func singleLine(s string) string { + s = sanitize(s) + if strings.ContainsAny(s, "\n\t") { + s = strings.NewReplacer("\n", " ", "\t", " ").Replace(s) + } + return s +} + +func isUnsafe(r rune) bool { + return unicode.IsControl(r) && r != '\n' && r != '\t' +} + +// sanitizeJSON cleans every string and key in a decoded JSON value. +func sanitizeJSON(v any) any { + switch x := v.(type) { + case string: + return sanitize(x) + case []any: + for i := range x { + x[i] = sanitizeJSON(x[i]) + } + case map[string]any: + var dirty []string + for k, val := range x { + x[k] = sanitizeJSON(val) + if sanitize(k) != k { + dirty = append(dirty, k) + } + } + // "ab" and "a\x00b" both clean to "ab"; number the collision rather than drop a field. + sort.Strings(dirty) + for _, k := range dirty { + clean := sanitize(k) + for n := 2; ; n++ { + if _, taken := x[clean]; !taken { + break + } + clean = fmt.Sprintf("%s (%d)", sanitize(k), n) + } + x[clean] = x[k] + delete(x, k) + } + } + return v +} diff --git a/internal/output/style.go b/internal/output/style.go new file mode 100644 index 0000000..6c8a9bc --- /dev/null +++ b/internal/output/style.go @@ -0,0 +1,22 @@ +package output + +import "github.com/charmbracelet/lipgloss" + +// Brand palette. Orange, not red, for negatives: traffic-light red is website-only. +const ( + omniPink = lipgloss.Color("#FF5FA2") + orange = lipgloss.Color("#FF7B3A") + darkGray = lipgloss.Color("#818181") + midGray = lipgloss.Color("#BABABA") +) + +var ( + styleDim = lipgloss.NewStyle().Foreground(darkGray) + styleBar = lipgloss.NewStyle().Foreground(omniPink) + styleNeg = lipgloss.NewStyle().Foreground(orange) + styleBorder = lipgloss.NewStyle().Foreground(darkGray) + styleHeader = lipgloss.NewStyle().Padding(0, 1).Bold(true).Foreground(omniPink) + styleCell = lipgloss.NewStyle().Padding(0, 1) + styleNum = lipgloss.NewStyle().Padding(0, 1).Align(lipgloss.Right) + styleMuted = lipgloss.NewStyle().Padding(0, 1).Foreground(midGray) +) diff --git a/internal/result/bench_test.go b/internal/result/bench_test.go new file mode 100644 index 0000000..de1aab0 --- /dev/null +++ b/internal/result/bench_test.go @@ -0,0 +1,62 @@ +package result + +import ( + "bytes" + "encoding/base64" + "encoding/json" + "fmt" + "testing" + + "github.com/apache/arrow-go/v18/arrow" + "github.com/apache/arrow-go/v18/arrow/array" + "github.com/apache/arrow-go/v18/arrow/ipc" + "github.com/apache/arrow-go/v18/arrow/memory" +) + +// StreamOfSize builds a realistic job line: one string dimension, one +// int measure, one float measure, n rows. +func StreamOfSize(n int) []byte { + mem := memory.NewGoAllocator() + schema := arrow.NewSchema([]arrow.Field{ + {Name: "e.country", Type: arrow.BinaryTypes.String, Nullable: true}, + {Name: "e.sessions", Type: arrow.PrimitiveTypes.Int64, Nullable: true}, + {Name: "e.pct", Type: arrow.PrimitiveTypes.Float64, Nullable: true}, + }, nil) + b := array.NewRecordBuilder(mem, schema) + defer b.Release() + for i := 0; i < n; i++ { + b.Field(0).(*array.StringBuilder).Append(fmt.Sprintf("country-%d", i%200)) + b.Field(1).(*array.Int64Builder).Append(int64(i * 7)) + b.Field(2).(*array.Float64Builder).Append(float64(i%100) / 100) + } + rec := b.NewRecordBatch() + defer rec.Release() + var buf bytes.Buffer + w := ipc.NewWriter(&buf, ipc.WithSchema(schema)) + _ = w.Write(rec) + _ = w.Close() + line, _ := json.Marshal(map[string]any{ + "job_id": "j", "status": "COMPLETE", + "summary": map[string]any{"fields": map[string]any{ + "e.country": map[string]any{"label": "Country", "is_dimension": true}, + "e.sessions": map[string]any{"label": "Sessions", "format": map[string]any{"value": "NUMBER_0"}}, + "e.pct": map[string]any{"label": "Pct", "format": map[string]any{"value": "percent"}}, + }}, + "result": base64.StdEncoding.EncodeToString(buf.Bytes()), + }) + return append(append([]byte(`{"jobs_submitted":{"j":"r"}}`+"\n"), line...), []byte("\n"+`{"remaining_job_ids":[]}`+"\n")...) +} + +func BenchmarkParse(b *testing.B) { + for _, n := range []int{100, 1000, 10000, 100000} { + data := StreamOfSize(n) + b.Run(fmt.Sprintf("rows=%d/bytes=%d", n, len(data)), func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + if _, err := Parse(data); err != nil { + b.Fatal(err) + } + } + }) + } +} diff --git a/internal/result/decimal.go b/internal/result/decimal.go new file mode 100644 index 0000000..bb0c6b0 --- /dev/null +++ b/internal/result/decimal.go @@ -0,0 +1,46 @@ +package result + +import ( + "math/big" + "strconv" + "strings" +) + +// Decimal is an exact number, Coef × 10^-Scale: a warehouse decimal or a uint64 past int64. +type Decimal struct { + Coef *big.Int + Scale int32 // never negative +} + +// newDecimal folds a negative scale into Coef, and returns int64 when the value fits one. +func newDecimal(coef *big.Int, scale int32) any { + if scale < 0 { + coef = new(big.Int).Mul(coef, new(big.Int).Exp(big.NewInt(10), big.NewInt(int64(-scale)), nil)) + scale = 0 + } + if scale == 0 && coef.IsInt64() { + return coef.Int64() + } + return Decimal{Coef: coef, Scale: scale} +} + +func (d Decimal) String() string { + digits := d.Coef.String() + sign := "" + if strings.HasPrefix(digits, "-") { + sign, digits = "-", digits[1:] + } + scale := int(d.Scale) + if scale == 0 { + return sign + digits + } + if len(digits) <= scale { + digits = strings.Repeat("0", scale-len(digits)+1) + digits + } + return sign + digits[:len(digits)-scale] + "." + digits[len(digits)-scale:] +} + +func (d Decimal) Float64() float64 { + f, _ := strconv.ParseFloat(d.String(), 64) + return f +} diff --git a/internal/result/pivot.go b/internal/result/pivot.go new file mode 100644 index 0000000..bd5b66b --- /dev/null +++ b/internal/result/pivot.go @@ -0,0 +1,238 @@ +package result + +import ( + "cmp" + "fmt" + "math/big" + "strings" + "time" +) + +// Pivoted is a Set reshaped as the query's pivots ask: one row per distinct +// value of the remaining dimensions, one column per pivot value and measure. +type Pivoted struct { + RowDims []int // column indices of the dimensions that stay as rows + PivotDims []int // column indices pivoted into column headers + Measures []int // column indices filling the cells + Keys [][]any // pivot values, one tuple per pivot column, in display order + Rows []PivotRow + // Omitted counts pivot values dropped by the query's column limit. + Omitted int +} + +// PivotRow is one row of a pivoted result. +type PivotRow struct { + Dims []any // values of RowDims + Cells [][]any // [key][measure]; nil where the combination has no row +} + +// Pivot reshapes the set, or returns nil when it isn't pivoted: no pivot +// field among its columns, or no measure to spread across them. +func (s *Set) Pivot() *Pivoted { + if len(s.Pivots) == 0 { + return nil + } + pivoted := map[string]bool{} + for _, name := range s.Pivots { + pivoted[name] = true + } + p := &Pivoted{} + for i, c := range s.Columns { + switch { + case pivoted[c.Name]: + p.PivotDims = append(p.PivotDims, i) + case c.IsDimension: + p.RowDims = append(p.RowDims, i) + default: + p.Measures = append(p.Measures, i) + } + } + if len(p.PivotDims) == 0 || len(p.Measures) == 0 { + return nil + } + + tuples := map[string][]any{} + rowIndex := map[string]int{} + var rowKeys []string + seqs := map[string][]string{} // row key -> its pivot keys, in stream order + type cell struct{ row, key string } + values := map[cell][]any{} + + for _, r := range s.Rows { + rk := tupleKey(r, p.RowDims) + pk := tupleKey(r, p.PivotDims) + if _, ok := rowIndex[rk]; !ok { + rowIndex[rk] = len(p.Rows) + rowKeys = append(rowKeys, rk) + p.Rows = append(p.Rows, PivotRow{Dims: pick(r, p.RowDims)}) + } + if _, ok := tuples[pk]; !ok { + tuples[pk] = pick(r, p.PivotDims) + } + seqs[rk] = append(seqs[rk], pk) + values[cell{rk, pk}] = pick(r, p.Measures) + } + desc := make([]bool, len(p.PivotDims)) + for i, c := range p.PivotDims { + desc[i] = s.Descending[s.Columns[c].Name] + } + order := mergeOrder(rowKeys, seqs, func(a, b string) bool { + return compareTuples(tuples[a], tuples[b], desc) < 0 + }) + + if s.ColumnLimit > 0 && len(order) > s.ColumnLimit { + p.Omitted = len(order) - s.ColumnLimit + order = order[:s.ColumnLimit] + } + for _, pk := range order { + p.Keys = append(p.Keys, tuples[pk]) + } + for i := range p.Rows { + p.Rows[i].Cells = make([][]any, len(order)) + for k, pk := range order { + p.Rows[i].Cells[k] = values[cell{rowKeys[i], pk}] + } + } + return p +} + +func pick(row []any, idx []int) []any { + out := make([]any, len(idx)) + for i, c := range idx { + out[i] = row[c] + } + return out +} + +// tupleKey identifies a tuple of cell values; the type is part of the key so +// the string "1" and the number 1 stay distinct, and the length so a string +// can't impersonate a separator. +func tupleKey(row []any, idx []int) string { + var b strings.Builder + for _, c := range idx { + v := fmt.Sprint(row[c]) + fmt.Fprintf(&b, "%T:%d:%s", row[c], len(v), v) + } + return b.String() +} + +// mergeOrder lays out pivot values. Each row group lists its values in the +// order the API sorted them; those orders are merged, and values they don't +// relate (never in the same group) fall back to the pivot fields' own order. +func mergeOrder(rowKeys []string, seqs map[string][]string, less func(a, b string) bool) []string { + var nodes []string + indegree := map[string]int{} + next := map[string]map[string]bool{} + for _, rk := range rowKeys { + seq := seqs[rk] + for j, pk := range seq { + if _, ok := indegree[pk]; !ok { + indegree[pk] = 0 + nodes = append(nodes, pk) + } + if j == 0 || seq[j-1] == pk || next[seq[j-1]][pk] { + continue + } + if next[seq[j-1]] == nil { + next[seq[j-1]] = map[string]bool{} + } + next[seq[j-1]][pk] = true + indegree[pk]++ + } + } + + order := make([]string, 0, len(nodes)) + done := map[string]bool{} + for len(order) < len(nodes) { + // The least ready value; if the groups' orders conflict, none is + // ready, and the least remaining value breaks the cycle. + best := "" + for _, ready := range []bool{true, false} { + for _, n := range nodes { + if done[n] || (ready && indegree[n] > 0) { + continue + } + if best == "" || less(n, best) { + best = n + } + } + if best != "" { + break + } + } + done[best] = true + order = append(order, best) + for n := range next[best] { + indegree[n]-- + } + } + return order +} + +// compareTuples orders pivot values as their fields sort: numbers and times +// by value, text lexically, nulls last; desc flips a field. +func compareTuples(a, b []any, desc []bool) int { + for i := range a { + c := compareValues(a[i], b[i]) + if desc[i] { + c = -c + } + if c != 0 { + return c + } + } + return 0 +} + +func compareValues(a, b any) int { + switch { + case a == nil && b == nil: + return 0 + case a == nil: + return 1 + case b == nil: + return -1 + } + _, aDec := a.(Decimal) + _, bDec := b.(Decimal) + if aDec || bDec { + if x, ok := exactRat(a); ok { + if y, ok := exactRat(b); ok { + return x.Cmp(y) + } + } + } + if x, ok := number(a); ok { + if y, ok := number(b); ok { + return cmp.Compare(x, y) + } + } + if x, ok := a.(time.Time); ok { + if y, ok := b.(time.Time); ok { + return x.Compare(y) + } + } + return strings.Compare(fmt.Sprint(a), fmt.Sprint(b)) +} + +func number(v any) (float64, bool) { + switch x := v.(type) { + case int64: + return float64(x), true + case float64: + return x, true + case Decimal: + return x.Float64(), true + } + return 0, false +} + +func exactRat(v any) (*big.Rat, bool) { + switch x := v.(type) { + case int64: + return new(big.Rat).SetInt64(x), true + case Decimal: + return new(big.Rat).SetFrac(x.Coef, new(big.Int).Exp(big.NewInt(10), big.NewInt(int64(x.Scale)), nil)), true + } + return nil, false +} diff --git a/internal/result/pivot_test.go b/internal/result/pivot_test.go new file mode 100644 index 0000000..e95e0e2 --- /dev/null +++ b/internal/result/pivot_test.go @@ -0,0 +1,134 @@ +package result + +import ( + "reflect" + "testing" +) + +// Region × stage, total amount pivoted on stage, as the stream sends it: +// long form, sorted by region then stage, EMEA with no Negotiation deals. +func pipelineSet() *Set { + return &Set{ + Columns: []Column{ + {Name: "deals.region", Label: "Region", IsDimension: true}, + {Name: "deals.stage", Label: "Stage", IsDimension: true}, + {Name: "deals.total_amount", Label: "Total amount", DataType: "NUMBER"}, + }, + Rows: [][]any{ + {"AMER", "Lost", int64(13)}, + {"AMER", "Negotiation", int64(1)}, + {"AMER", "Won", int64(3)}, + {"EMEA", "Lost", int64(8)}, + {"EMEA", "Won", int64(2)}, + }, + Pivots: []string{"deals.stage"}, + } +} + +func TestPivot_Reshapes(t *testing.T) { + p := pipelineSet().Pivot() + if p == nil { + t.Fatal("expected a pivot") + } + if !reflect.DeepEqual(p.RowDims, []int{0}) || !reflect.DeepEqual(p.PivotDims, []int{1}) || !reflect.DeepEqual(p.Measures, []int{2}) { + t.Fatalf("roles: rows %v pivots %v measures %v", p.RowDims, p.PivotDims, p.Measures) + } + if want := [][]any{{"Lost"}, {"Negotiation"}, {"Won"}}; !reflect.DeepEqual(p.Keys, want) { + t.Errorf("keys = %v, want %v", p.Keys, want) + } + if len(p.Rows) != 2 || p.Rows[1].Dims[0] != "EMEA" { + t.Fatalf("rows = %+v", p.Rows) + } + if p.Rows[1].Cells[1] != nil { + t.Errorf("EMEA has no Negotiation row, got %v", p.Rows[1].Cells[1]) + } + if p.Rows[1].Cells[2][0] != int64(2) { + t.Errorf("EMEA won = %v", p.Rows[1].Cells[2]) + } +} + +// A value the first row group lacks lands where the later group puts it, +// not at the end. +func TestPivot_OrderMergesAcrossGroups(t *testing.T) { + set := pipelineSet() + set.Rows = [][]any{ + {"AMER", "Lost", int64(13)}, + {"AMER", "Won", int64(3)}, + {"EMEA", "Lost", int64(8)}, + {"EMEA", "Negotiation", int64(1)}, + {"EMEA", "Won", int64(2)}, + {"APAC", "Early", int64(1)}, + {"APAC", "Lost", int64(1)}, + } + want := [][]any{{"Early"}, {"Lost"}, {"Negotiation"}, {"Won"}} + if got := set.Pivot().Keys; !reflect.DeepEqual(got, want) { + t.Errorf("keys = %v, want %v", got, want) + } +} + +func TestPivot_ColumnLimit(t *testing.T) { + set := pipelineSet() + set.ColumnLimit = 2 + p := set.Pivot() + if len(p.Keys) != 2 || p.Omitted != 1 || len(p.Rows[0].Cells) != 2 { + t.Errorf("keys %v omitted %d", p.Keys, p.Omitted) + } +} + +func TestPivot_NotPivoted(t *testing.T) { + set := pipelineSet() + set.Pivots = nil + if set.Pivot() != nil { + t.Error("no pivots: no pivot") + } + set.Pivots = []string{"deals.nope"} + if set.Pivot() != nil { + t.Error("a pivot on a field not in the result: no pivot") + } + set = pipelineSet() + set.Columns = set.Columns[:2] + for i := range set.Rows { + set.Rows[i] = set.Rows[i][:2] + } + if set.Pivot() != nil { + t.Error("nothing to spread across the columns: no pivot") + } +} + +// Values no row group relates fall back to their own order: row A has Q2 +// and Q4, row B Q1 and Q3. +func TestPivot_UnrelatedValuesUseValueOrder(t *testing.T) { + set := pipelineSet() + set.Rows = [][]any{ + {"A", "Q2", int64(1)}, + {"A", "Q4", int64(1)}, + {"B", "Q1", int64(1)}, + {"B", "Q3", int64(1)}, + } + want := [][]any{{"Q1"}, {"Q2"}, {"Q3"}, {"Q4"}} + if got := set.Pivot().Keys; !reflect.DeepEqual(got, want) { + t.Errorf("keys = %v, want %v", got, want) + } + set.Descending = map[string]bool{"deals.stage": true} + set.Rows = [][]any{ + {"A", "Q4", int64(1)}, + {"A", "Q2", int64(1)}, + {"B", "Q3", int64(1)}, + {"B", "Q1", int64(1)}, + } + want = [][]any{{"Q4"}, {"Q3"}, {"Q2"}, {"Q1"}} + if got := set.Pivot().Keys; !reflect.DeepEqual(got, want) { + t.Errorf("descending: keys = %v, want %v", got, want) + } +} + +func TestTupleKey_StringsCannotCollide(t *testing.T) { + a := []any{"a\x00string:b", "c"} + b := []any{"a", "b\x00string:c"} + if tupleKey(a, []int{0, 1}) == tupleKey(b, []int{0, 1}) { + t.Error("distinct tuples produced the same key") + } + if tupleKey([]any{"1"}, []int{0}) == tupleKey([]any{int64(1)}, []int{0}) { + t.Error("the string \"1\" and the number 1 should stay distinct") + } +} diff --git a/internal/result/result.go b/internal/result/result.go new file mode 100644 index 0000000..1a15a5b --- /dev/null +++ b/internal/result/result.go @@ -0,0 +1,338 @@ +// Package result decodes the query/run and query/wait NDJSON streams: rows +// in query order plus the model's metadata for each column. +package result + +import ( + "bufio" + "bytes" + "encoding/base64" + "encoding/json" + "errors" + "fmt" + "math" + "math/big" + "strings" + "unicode" + + "github.com/apache/arrow-go/v18/arrow" + "github.com/apache/arrow-go/v18/arrow/array" + "github.com/apache/arrow-go/v18/arrow/ipc" +) + +// Column is one field of a result set. +type Column struct { + Name string // fully qualified, e.g. events_ext.sessions + Label string + IsDimension bool + DataType string // NUMBER, STRING, TIMESTAMP, ... + Format string // NUMBER_0, percent, ... ; empty when the model sets none +} + +// Set is one job's decoded result. +type Set struct { + JobID string + Columns []Column + Rows [][]any // string, int64, float64, Decimal, bool, time.Time, or nil + // Pivots names the columns the query pivots on. The stream carries rows + // in long form either way; Pivot reshapes them. + Pivots []string + // ColumnLimit caps the pivot columns shown, as the query sets it; 0 means none. + ColumnLimit int + // Descending lists the columns the query sorts high to low. + Descending map[string]bool +} + +// Stream is one parsed query/run or query/wait response. +type Stream struct { + Sets []*Set + // Remaining lists jobs still running; poll query/wait with them until empty. + Remaining []string + // Failures lists jobs that did not complete. One bad job doesn't void the + // others: the caller renders what decoded and reports these alongside. + Failures []Failure +} + +// Failure is one job the API could not complete, or whose result would not decode. +type Failure struct { + JobID string + Status string + Message string +} + +func (f Failure) Error() string { + if f.Status == "" { + return fmt.Sprintf("query job %s: %s", f.JobID, f.Message) + } + return fmt.Sprintf("query job %s %s: %s", f.JobID, f.Status, f.Message) +} + +// Err reports the stream's failed jobs as a single error, or nil if none failed. +func (s *Stream) Err() error { + switch len(s.Failures) { + case 0: + return nil + case 1: + return s.Failures[0] + } + msgs := make([]string, len(s.Failures)) + for i, f := range s.Failures { + msgs[i] = f.Error() + } + return errors.New(strings.Join(msgs, "; ")) +} + +type line struct { + JobsSubmitted map[string]string `json:"jobs_submitted"` + JobID string `json:"job_id"` + Status string `json:"status"` + Summary *summary `json:"summary"` + Query *struct { + ModelJob struct { + Fields []string `json:"fields"` + Pivots []string `json:"pivots"` + ColumnLimit int `json:"column_limit"` + Sorts []struct { + ColumnName string `json:"column_name"` + SortDescending bool `json:"sort_descending"` + } `json:"sorts"` + } `json:"model_job"` + } `json:"query"` + Result string `json:"result"` + ErrorType string `json:"error_type"` + ErrorMessage string `json:"error_message"` + Error json.RawMessage `json:"error"` + RemainingJobIDs []string `json:"remaining_job_ids"` +} + +type summary struct { + Fields map[string]fieldMeta `json:"fields"` +} + +type fieldMeta struct { + FieldName string `json:"field_name"` + Label string `json:"label"` + IsDimension bool `json:"is_dimension"` + DataType string `json:"data_type"` + Format *struct { + Value string `json:"value"` + } `json:"format"` +} + +// Parse decodes a stream body. A job that failed or would not decode is +// collected in Stream.Failures rather than aborting the whole stream, so the +// jobs that did complete are still rendered. +func Parse(data []byte) (*Stream, error) { + var st Stream + sc := bufio.NewScanner(bytes.NewReader(data)) + sc.Buffer(nil, 1<<30) + for sc.Scan() { + raw := bytes.TrimSpace(sc.Bytes()) + if len(raw) == 0 { + continue + } + var l line + if err := json.Unmarshal(raw, &l); err != nil { + return nil, fmt.Errorf("query stream: %w", err) + } + switch { + case l.RemainingJobIDs != nil: + st.Remaining = l.RemainingJobIDs + case l.JobID != "": + if l.Status != "COMPLETE" { + st.Failures = append(st.Failures, Failure{JobID: l.JobID, Status: l.Status, Message: jobError(l)}) + continue + } + set, err := decodeJob(l) + if err != nil { + st.Failures = append(st.Failures, Failure{JobID: l.JobID, Message: err.Error()}) + continue + } + st.Sets = append(st.Sets, set) + } + } + if err := sc.Err(); err != nil { + return nil, fmt.Errorf("query stream: %w", err) + } + return &st, nil +} + +func jobError(l line) string { + if l.ErrorMessage != "" { + if l.ErrorType != "" { + return l.ErrorType + ": " + l.ErrorMessage + } + return l.ErrorMessage + } + if len(l.Error) == 0 { + return "no detail" + } + var s string + if json.Unmarshal(l.Error, &s) == nil { + return s + } + var obj map[string]any + if json.Unmarshal(l.Error, &obj) == nil { + for _, k := range []string{"message", "detail", "error"} { + if v, ok := obj[k].(string); ok && v != "" { + return v + } + } + } + return string(l.Error) +} + +func decodeJob(l line) (*Set, error) { + if l.Result == "" { + return nil, fmt.Errorf("job carries no result (a planOnly query has none to render; use --format json)") + } + raw, err := base64.StdEncoding.DecodeString(l.Result) + if err != nil { + return nil, fmt.Errorf("result is not base64: %w", err) + } + rd, err := ipc.NewReader(bytes.NewReader(raw)) + if err != nil { + return nil, fmt.Errorf("result is not an Arrow stream: %w", err) + } + defer rd.Release() + + // The batch carries extra columns (sort keys, primary key, "__raw" twins), + // so the query's own field list decides which show and in what order. + schema := rd.Schema() + var picked []int + if l.Query != nil && len(l.Query.ModelJob.Fields) > 0 { + for _, name := range l.Query.ModelJob.Fields { + idx := schema.FieldIndices(name) + if len(idx) == 0 { + // A partial match would silently drop columns; fall back to + // the summary's field set, which describes what arrived. + picked = nil + break + } + picked = append(picked, idx[0]) + } + } + if len(picked) == 0 { + described := l.Summary != nil && len(l.Summary.Fields) > 0 + for i, f := range schema.Fields() { + if !described { + picked = append(picked, i) + } else if _, ok := l.Summary.Fields[f.Name]; ok { + picked = append(picked, i) + } + } + } + if len(picked) == 0 { + return nil, fmt.Errorf("result has no columns the query asked for") + } + + set := &Set{JobID: l.JobID} + if l.Query != nil { + set.Pivots = l.Query.ModelJob.Pivots + set.ColumnLimit = l.Query.ModelJob.ColumnLimit + for _, s := range l.Query.ModelJob.Sorts { + if s.SortDescending { + if set.Descending == nil { + set.Descending = map[string]bool{} + } + set.Descending[s.ColumnName] = true + } + } + } + for _, i := range picked { + f := schema.Field(i) + col := Column{Name: f.Name, Label: f.Name} + if l.Summary != nil { + if m, ok := l.Summary.Fields[f.Name]; ok { + col.Label = m.Label + if col.Label == "" && m.FieldName != "" { + col.Label = Humanize(m.FieldName) + } + col.IsDimension = m.IsDimension + col.DataType = m.DataType + if m.Format != nil { + col.Format = m.Format.Value + } + } + } + set.Columns = append(set.Columns, col) + } + + for rd.Next() { + rec := rd.RecordBatch() + n := int(rec.NumRows()) + for r := 0; r < n; r++ { + row := make([]any, len(picked)) + for j, c := range picked { + row[j] = value(rec.Column(c), r) + } + set.Rows = append(set.Rows, row) + } + } + if err := rd.Err(); err != nil { + return nil, fmt.Errorf("reading Arrow batches: %w", err) + } + return set, nil +} + +// Humanize turns "avg_scroll_depth" into "Avg Scroll Depth", as Omni labels unlabelled fields. +func Humanize(name string) string { + parts := strings.FieldsFunc(name, func(r rune) bool { return r == '_' || r == '-' || r == ' ' }) + for i, p := range parts { + r := []rune(p) + r[0] = unicode.ToUpper(r[0]) + parts[i] = string(r) + } + return strings.Join(parts, " ") +} + +// value converts one Arrow cell: ints to int64, floats to float64, decimals +// (and unsigned ints past int64) to an exact Decimal, anything unhandled to +// Arrow's string form. +func value(col arrow.Array, i int) any { + if col.IsNull(i) { + return nil + } + switch a := col.(type) { + case *array.String: + return a.Value(i) + case *array.LargeString: + return a.Value(i) + case *array.Boolean: + return a.Value(i) + case *array.Int8: + return int64(a.Value(i)) + case *array.Int16: + return int64(a.Value(i)) + case *array.Int32: + return int64(a.Value(i)) + case *array.Int64: + return a.Value(i) + case *array.Uint8: + return int64(a.Value(i)) + case *array.Uint16: + return int64(a.Value(i)) + case *array.Uint32: + return int64(a.Value(i)) + case *array.Uint64: + if v := a.Value(i); v > math.MaxInt64 { + return Decimal{Coef: new(big.Int).SetUint64(v)} + } + return int64(a.Value(i)) + case *array.Float32: + return float64(a.Value(i)) + case *array.Float64: + return a.Value(i) + case *array.Decimal128: + return newDecimal(a.Value(i).BigInt(), a.DataType().(*arrow.Decimal128Type).Scale) + case *array.Decimal256: + return newDecimal(a.Value(i).BigInt(), a.DataType().(*arrow.Decimal256Type).Scale) + case *array.Timestamp: + unit := a.DataType().(*arrow.TimestampType).Unit + return a.Value(i).ToTime(unit).UTC() + case *array.Date32: + return a.Value(i).ToTime().UTC() + case *array.Date64: + return a.Value(i).ToTime().UTC() + } + return col.ValueStr(i) +} diff --git a/internal/result/result_test.go b/internal/result/result_test.go new file mode 100644 index 0000000..155730b --- /dev/null +++ b/internal/result/result_test.go @@ -0,0 +1,254 @@ +package result + +import ( + "bytes" + "encoding/base64" + "encoding/json" + "math" + "math/big" + "strings" + "testing" + "time" + + "github.com/apache/arrow-go/v18/arrow" + "github.com/apache/arrow-go/v18/arrow/array" + "github.com/apache/arrow-go/v18/arrow/decimal128" + "github.com/apache/arrow-go/v18/arrow/ipc" + "github.com/apache/arrow-go/v18/arrow/memory" +) + +// arrowPayload builds the base64 Arrow IPC stream the API puts in a job +// line: country, sessions, engaged %, plus a date column. +func arrowPayload(t *testing.T) string { + t.Helper() + mem := memory.NewGoAllocator() + schema := arrow.NewSchema([]arrow.Field{ + {Name: "events_ext.country", Type: arrow.BinaryTypes.String, Nullable: true}, + {Name: "events_ext.sessions", Type: arrow.PrimitiveTypes.Int64, Nullable: true}, + {Name: "events_ext.engaged_sessions_percent", Type: arrow.PrimitiveTypes.Float64, Nullable: true}, + {Name: "events_ext.event_timestamp[date]", Type: arrow.FixedWidthTypes.Date32, Nullable: true}, + }, nil) + + b := array.NewRecordBuilder(mem, schema) + defer b.Release() + b.Field(0).(*array.StringBuilder).AppendValues([]string{"United States", "Ireland"}, nil) + b.Field(1).(*array.Int64Builder).AppendValues([]int64{12526, 838}, nil) + b.Field(2).(*array.Float64Builder).AppendValues([]float64{0.3992, 0}, []bool{true, false}) + d := arrow.Date32FromTime(time.Date(2026, 9, 1, 0, 0, 0, 0, time.UTC)) + b.Field(3).(*array.Date32Builder).AppendValues([]arrow.Date32{d, d}, nil) + rec := b.NewRecordBatch() + defer rec.Release() + + var buf bytes.Buffer + w := ipc.NewWriter(&buf, ipc.WithSchema(schema), ipc.WithAllocator(mem)) + if err := w.Write(rec); err != nil { + t.Fatal(err) + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + return base64.StdEncoding.EncodeToString(buf.Bytes()) +} + +func jobLine(t *testing.T, id, status string, extra map[string]any) string { + t.Helper() + l := map[string]any{ + "job_id": id, + "status": status, + "summary": map[string]any{ + "fields": map[string]any{ + "events_ext.country": map[string]any{"label": "Country", "is_dimension": true, "data_type": "STRING"}, + "events_ext.sessions": map[string]any{"label": "Sessions", "data_type": "NUMBER", "format": map[string]any{"value": "NUMBER_0"}}, + "events_ext.engaged_sessions_percent": map[string]any{"label": "Engaged Sessions %", "data_type": "NUMBER", "format": map[string]any{"value": "percent"}}, + "events_ext.event_timestamp[date]": map[string]any{"label": "Date", "is_dimension": true, "data_type": "DATE"}, + }, + }, + } + if status == "COMPLETE" { + l["result"] = arrowPayload(t) + } + for k, v := range extra { + l[k] = v + } + raw, _ := json.Marshal(l) + return string(raw) +} + +func TestParse_DecodesAJob(t *testing.T) { + body := strings.Join([]string{ + `{"jobs_submitted":{"j1":"r1"}}`, + jobLine(t, "j1", "COMPLETE", nil), + `{"remaining_job_ids":[],"timed_out":"false"}`, + }, "\n") + st, err := Parse([]byte(body)) + if err != nil { + t.Fatalf("Parse: %v", err) + } + if len(st.Sets) != 1 || len(st.Remaining) != 0 { + t.Fatalf("expected one set and nothing remaining, got %d / %v", len(st.Sets), st.Remaining) + } + set := st.Sets[0] + + // Column order is the schema's (the query's), with the model's metadata + // looked up by name. + wantCols := []Column{ + {Name: "events_ext.country", Label: "Country", IsDimension: true, DataType: "STRING"}, + {Name: "events_ext.sessions", Label: "Sessions", DataType: "NUMBER", Format: "NUMBER_0"}, + {Name: "events_ext.engaged_sessions_percent", Label: "Engaged Sessions %", DataType: "NUMBER", Format: "percent"}, + {Name: "events_ext.event_timestamp[date]", Label: "Date", IsDimension: true, DataType: "DATE"}, + } + for i, want := range wantCols { + if set.Columns[i] != want { + t.Errorf("column %d = %+v, want %+v", i, set.Columns[i], want) + } + } + + if len(set.Rows) != 2 { + t.Fatalf("expected 2 rows, got %d", len(set.Rows)) + } + if set.Rows[0][0] != "United States" || set.Rows[0][1] != int64(12526) || set.Rows[0][2] != 0.3992 { + t.Errorf("row 0 = %v", set.Rows[0]) + } + if set.Rows[1][2] != nil { + t.Errorf("null cell should decode as nil, got %v", set.Rows[1][2]) + } + if d, ok := set.Rows[0][3].(time.Time); !ok || d.Format("2006-01-02") != "2026-09-01" { + t.Errorf("date cell = %v", set.Rows[0][3]) + } +} + +func TestHumanize(t *testing.T) { + for in, want := range map[string]string{"avg_scroll_depth": "Avg Scroll Depth", "amount": "Amount", "is_won": "Is Won", "état_civil": "État Civil"} { + if got := Humanize(in); got != want { + t.Errorf("Humanize(%q) = %q, want %q", in, got, want) + } + } +} + +func TestParse_TimedOutStreamReportsRemaining(t *testing.T) { + body := `{"jobs_submitted":{"j1":"r1"}}` + "\n" + `{"remaining_job_ids":["j1"],"timed_out":"true"}` + "\n" + st, err := Parse([]byte(body)) + if err != nil { + t.Fatal(err) + } + if len(st.Sets) != 0 || len(st.Remaining) != 1 || st.Remaining[0] != "j1" { + t.Errorf("got sets=%d remaining=%v", len(st.Sets), st.Remaining) + } +} + +func TestParse_FailedJobIsAFailure(t *testing.T) { + // The documented shape: error_type and error_message on the job line. + body := jobLine(t, "j1", "ERROR", map[string]any{"error_type": "PLAN", "error_message": `No such view "order_items"`}) + st, err := Parse([]byte(body)) + if err != nil { + t.Fatal(err) + } + if err := st.Err(); err == nil || !strings.Contains(err.Error(), `PLAN: No such view "order_items"`) { + t.Errorf("expected the API's message, got %v", err) + } + body = jobLine(t, "j1", "ERROR", map[string]any{"error": map[string]any{"message": "token expired"}}) + st, err = Parse([]byte(body)) + if err != nil { + t.Fatal(err) + } + if err := st.Err(); err == nil || !strings.Contains(err.Error(), "token expired") { + t.Errorf("structured error should still surface, got %v", err) + } +} + +func TestParse_OneFailedJobKeepsTheOthers(t *testing.T) { + body := jobLine(t, "j1", "COMPLETE", nil) + "\n" + + jobLine(t, "j2", "ERROR", map[string]any{"error_message": "boom"}) + st, err := Parse([]byte(body)) + if err != nil { + t.Fatal(err) + } + if len(st.Sets) != 1 || st.Sets[0].JobID != "j1" { + t.Errorf("the completed job should still decode, got %d sets", len(st.Sets)) + } + if err := st.Err(); err == nil || !strings.Contains(err.Error(), "boom") { + t.Errorf("the failed job should still be reported, got %v", err) + } +} + +func TestParse_UnknownFieldKeepsItsName(t *testing.T) { + // A column the summary doesn't describe still renders, labelled by name. + body := jobLine(t, "j1", "COMPLETE", map[string]any{"summary": map[string]any{"fields": map[string]any{}}}) + st, err := Parse([]byte(body)) + if err != nil { + t.Fatal(err) + } + if c := st.Sets[0].Columns[1]; c.Label != "events_ext.sessions" || c.Format != "" { + t.Errorf("undescribed column = %+v", c) + } +} + +func TestParse_CarriesPivots(t *testing.T) { + body := jobLine(t, "j1", "COMPLETE", map[string]any{"query": map[string]any{"model_job": map[string]any{ + "pivots": []string{"events_ext.event_timestamp[date]"}, + "column_limit": 50, + }}}) + st, err := Parse([]byte(body)) + if err != nil { + t.Fatal(err) + } + set := st.Sets[0] + if len(set.Pivots) != 1 || set.Pivots[0] != "events_ext.event_timestamp[date]" || set.ColumnLimit != 50 { + t.Errorf("pivots = %v, column limit = %d", set.Pivots, set.ColumnLimit) + } +} + +func TestValue_DecimalsAndLargeUnsignedStayExact(t *testing.T) { + mem := memory.NewGoAllocator() + + db := array.NewDecimal128Builder(mem, &arrow.Decimal128Type{Precision: 38, Scale: 2}) + defer db.Release() + big20, _ := new(big.Int).SetString("12345678901234567891", 10) + db.Append(decimal128.FromBigInt(big20)) + db.Append(decimal128.FromI64(9007199254740993)) + decimals := db.NewArray() + defer decimals.Release() + + whole := array.NewDecimal128Builder(mem, &arrow.Decimal128Type{Precision: 38, Scale: 0}) + defer whole.Release() + whole.Append(decimal128.FromI64(9007199254740993)) + wholes := whole.NewArray() + defer wholes.Release() + + ub := array.NewUint64Builder(mem) + defer ub.Release() + ub.AppendValues([]uint64{math.MaxUint64, 7}, nil) + uints := ub.NewArray() + defer uints.Release() + + for _, tc := range []struct { + col arrow.Array + i int + want any + }{ + {decimals, 0, "123456789012345678.91"}, + {decimals, 1, "90071992547409.93"}, + {wholes, 0, int64(9007199254740993)}, + {uints, 0, "18446744073709551615"}, + {uints, 1, int64(7)}, + } { + got := value(tc.col, tc.i) + if d, ok := got.(Decimal); ok { + got = d.String() + } + if got != tc.want { + t.Errorf("value(%s, %d) = %#v, want %#v", tc.col.DataType(), tc.i, got, tc.want) + } + } +} + +func TestCompareValues_DecimalsCompareExactly(t *testing.T) { + a := Decimal{Coef: big.NewInt(9007199254740993)} + b := Decimal{Coef: big.NewInt(9007199254740992)} + if compareValues(a, b) <= 0 || compareValues(b, a) >= 0 { + t.Error("decimals a float can't tell apart should still order") + } + if compareValues(Decimal{Coef: big.NewInt(150), Scale: 2}, int64(1)) <= 0 { + t.Error("1.50 should sort after 1") + } +} diff --git a/scripts/chart-demo.sh b/scripts/chart-demo.sh new file mode 100755 index 0000000..bb17432 --- /dev/null +++ b/scripts/chart-demo.sh @@ -0,0 +1,86 @@ +#!/usr/bin/env bash +# Walk through query rendering: tables, bar charts, and pivots. +# Read-only queries against a sales pipeline demo model. +# +# OMNI_PROFILE=my-profile MODEL_ID=... scripts/chart-demo.sh +# PAUSE=1 ... # wait for Enter between examples +# VIEW=... TOPIC=... # if your model names them differently +set -uo pipefail + +cd "$(dirname "$0")/.." +: "${OMNI_PROFILE:?set OMNI_PROFILE to the omni config profile to query with}" +: "${MODEL_ID:?set MODEL_ID to a model with the sales pipeline demo topic}" +PROFILE=$OMNI_PROFILE +M=$MODEL_ID +D=${VIEW:-apps_demos_sales_pipeline__deals} +T=${TOPIC:-sales_pipeline} + +[[ -x bin/omni ]] || make build >/dev/null + +bold=$'\e[1m' dim=$'\e[2m' reset=$'\e[0m' + +# q FIELDS [EXTRA_JSON] -> a query/run body +q() { + local fields="" f + for f in $1; do fields+="${fields:+,}\"$D.$f\""; done + printf '{"query":{"modelId":"%s","table":"%s","topic":"%s","fields":[%s],"limit":100%s}}' \ + "$M" "$D" "$T" "$fields" "${2:-}" +} +by_region=',"sorts":[{"column_name":"'$D'.region","sort_descending":false}]' +pivot_stage="$by_region"',"pivots":["'$D'.stage"]' + +step() { + echo + echo "${bold}━━ $1${reset}" + shift + echo "${dim}\$ omni $*${reset}" +} +pause() { [[ -n ${PAUSE:-} ]] && read -r -p "${dim}(enter)${reset}" _; return 0; } +run() { bin/omni --profile "$PROFILE" "$@"; local s=$?; [[ $s -ne 0 ]] && echo "${dim}exit $s${reset}"; pause; } + +echo "${bold}omni query rendering demo${reset} ${dim}profile=$PROFILE, terminal width ${COLUMNS:-$(tput cols)}${reset}" + +# ── One dimension, several measures ────────────────────────────────────────── +F1="region count total_amount won_amount win_rate" +step "1 dim × 4 measures: table" query run --format human +run query run --format human --body "$(q "$F1")" + +step "1 dim × 4 measures: chart (every measure, each on its own scale)" query run --chart +run query run --chart --body "$(q "$F1")" + +step "Two measures, picked by label and field" query run --chart --chart-value "Win rate",count +run query run --chart --chart-value "Win rate",count --body "$(q "$F1")" + + +# ── Two dimensions, two measures ───────────────────────────────────────────── +F2="region stage total_amount count" +step "2 dims × 2 measures: table" query run --format human +run query run --format human --body "$(q "$F2" "$by_region")" + +step "2 dims × 2 measures: chart (both dimensions label each row)" query run --chart +run query run --chart --body "$(q "$F2" "$by_region")" + +step "Row cap" query run --chart --chart-rows 5 +run query run --chart --chart-rows 5 --body "$(q "$F2" "$by_region")" + +# ── Pivots ─────────────────────────────────────────────────────────────────── +F3="region stage total_amount" +step "Pivot (region × stage, 1 measure): table, as the workbook shows it" query run --format human +run query run --format human --body "$(q "$F3" "$pivot_stage")" + +step "Pivot: chart (stages share one scale; widen the terminal for more columns)" query run --chart +run query run --chart --body "$(q "$F3" "$pivot_stage")" + +step "Pivot with 2 measures: table" query run --format human +run query run --format human --body "$(q "$F2" "$pivot_stage")" + +step "Pivot with 2 measures: chart, narrowed to count" query run --chart --chart-value count +run query run --chart --chart-value count --body "$(q "$F2" "$pivot_stage")" + +step "Pivot + workbook link (compare with the Omni app)" query run --format human --workbook +run query run --format human --workbook --body "$(q "$F3" "$pivot_stage")" + +# ── Pipes ──────────────────────────────────────────────────────────────────── +step "Piped: still draws" "query run --chart | cat" +bin/omni --profile "$PROFILE" query run --chart --chart-value total_amount --body "$(q "$F1")" | cat +pause