Pipe process failing for read_text & read_blob - but fine for read_csv & read_json

[baikal]

very nice extension!

I run into the following error when trying to just read plain output from shell commands:

$ duckdb 
DuckDB v1.4.4 (Andium) 6ddac802ff
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D load shellfs;
D from read_text('uname |');
Exception in ShellFileHandle destructor: {"exception_type":"IO","exception_message":"Pipe process exited abnormally code=141: uname |"}
┌──────────┬─────────┬───────┬──────────────────────────┐
│ filename │ content │ size  │      last_modified       │
│ varchar  │ varchar │ int64 │ timestamp with time zone │
├──────────┼─────────┼───────┼──────────────────────────┤
│ uname |  │         │   0   │ 1970-01-01 01:00:00+01   │
└──────────┴─────────┴───────┴──────────────────────────┘
D from read_blob('uname |');
Exception in ShellFileHandle destructor: {"exception_type":"IO","exception_message":"Pipe process exited abnormally code=141: uname |"}
┌──────────┬─────────┬───────┬──────────────────────────┐
│ filename │ content │ size  │      last_modified       │
│ varchar  │  blob   │ int64 │ timestamp with time zone │
├──────────┼─────────┼───────┼──────────────────────────┤
│ uname |  │         │   0   │ 1970-01-01 01:00:00+01   │
└──────────┴─────────┴───────┴──────────────────────────┘
D from read_csv('uname |');
┌─────────┐
│  Linux  │
│ varchar │
├─────────┤
│ 0 rows  │
└─────────┘
D from read_json('uname -rs | jq -R |');
┌────────────────────────┐
│          json          │
│        varchar         │
├────────────────────────┤
│ Linux 6.8.0-51-generic │
└────────────────────────┘

• shellfs version 4fef6d3
• on Ubuntu 24.04.1 LTS
• Linux 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec 5 13:09:44 UTC 2024 x86_64

All the same for uname, ps, whoami, /bin/ls and many more

[rustyconover]

Hey @baikal,

It seems like read_blob() and read_text() are causing issues but the others work great.

Rusty

[baikal]

Exactly - which is what I tried to document in the first place ...
And read_text() is the thing that would really help me for a project. In the meantime, back to writing temporary files, then.