map[key] fails when the key literal's type differs from the map key type (e.g. MAP(VARCHAR, ...) with m['a']) #26197
Description
Describe the bug
map[key] (get_field on a map) does not coerce the lookup key to the map's key type. Any map whose key type differs from the literal's type fails at execution time.
The most common case is a map declared in SQL with VARCHAR keys. Since VARCHAR maps to Utf8View by default, the map has Utf8View keys, while 'a' is a Utf8 literal. Integer keys hit the same error: MAP(INT, ...) with m[1], where the literal is Int64.
map_extract(m, 'a') and element_at(m, 'a') work on the same column, because map_extract coerces the key to the map key type in coerce_types.
To Reproduce
CREATE TABLE t (m MAP(VARCHAR, BIGINT)) AS VALUES (MAP {'a': 1, 'b': 2});
SELECT m['a'] FROM t;
-- Execution error: The key type Utf8 does not match the map key type Utf8View
SELECT map_extract(m, 'a') FROM t; -- [1]
SELECT element_at(m, 'a') FROM t; -- [1]
SELECT m[arrow_cast('a', 'Utf8View')] FROM t; -- 1
CREATE TABLE w (m MAP(INT, BIGINT)) AS VALUES (MAP {1: 10});
SELECT m[1] FROM w;
-- Execution error: The key type Int64 does not match the map key type Int32Nested maps fail the same way (b['a']['aa'] on MAP(VARCHAR, MAP(VARCHAR, INT))), as do filters such as WHERE m['a'] = 1.
With SET datafusion.sql_parser.map_string_types_to_utf8view = false, the VARCHAR case returns 1.
Expected behavior
m['a'] returns 1 and m[1] returns 10. The lookup key should be coerced to the map key type, the same way map_extract does it.
Additional context
- Tested on
mainat a4b8198. - Not a regression from perf: Unify, optimize
map[k]andmap_extract(key)#25201. The commit before it (28c0dec) fails on the same queries, withArrow error: Invalid argument error: Invalid comparison operation: Utf8 == Utf8View(andInt64 == Int32). perf: Unify, optimizemap[k]andmap_extract(key)#25201 only changed the error message. GetFieldFunc::coerce_typesreturns the argument types unchanged ("validation happens in return_field_from_args").return_field_from_argsdoes not check the map key type either. The key reachesmap_lookup(datafusion/functions/src/utils.rs), which requires the lookup key type to equal the map key type, ignoring dictionary encoding.MAP(BIGINT, ...)withm[1]andMAP(DOUBLE, ...)withm[1.5]work, because those literals already have the map's key type.- The existing map tests (
map.slt, and themap(int, int)/map(varchar, ...)table inddl.slt) never look up a key on a map column declared in SQL.
Willingness to contribute
I plan to contribute a fix for this bug independently
Activity
Ishaan400 commentedon Oct 11, 2026
take