Fallback to raw string value, when json field inside geojson isn't decodable

[jbylina]

Expected behavior and actual behavior.

I expected to successfully read the following geojson file by Fiona and leave the handling of mixed types geojson properties to lib user.

{ "type": "FeatureCollection", "features": [ {"type": "Feature", "properties": { "a": [ ]}, "geometry": { "type": "Point", "coordinates": [ [-74, 5]] } }, {"type": "Feature", "properties": { "a": ""}, "geometry": { "type": "Point", "coordinates": [ [-74, 5]] } } ] }

Actual behavior:

Traceback (most recent call last): File "<input>", line 1, in <module> File "fiona/ogrext.pyx", line 1996, in fiona.ogrext.Iterator.__next__ File "fiona/ogrext.pyx", line 668, in fiona.ogrext.FeatureBuilder.build File "fiona/ogrext.pyx", line 390, in fiona.ogrext.JSONField.get File "/Users/jacek/.pyenv/versions/3.12.7-debug/lib/python3.12/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jacek/.pyenv/versions/3.12.7-debug/lib/python3.12/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jacek/.pyenv/versions/3.12.7-debug/lib/python3.12/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 

Steps to reproduce the problem.

Run the following script using the latest Fiona.

from pathlib import Path import fiona Path("test.geojson").write_text("""\ { "type": "FeatureCollection", "features": [ {"type": "Feature", "properties": { "a": [ ]}, "geometry": { "type": "Point", "coordinates": [ [-74, 5]] } }, {"type": "Feature", "properties": { "a": ""}, "geometry": { "type": "Point", "coordinates": [ [-74, 5]] } } ] } """) list(fiona.open("test.geojson", "r"))

Operating system

macOS 15.1.1

Fiona and GDAL version and provenance

Fiona 1.10.1 from pip
GDAL 3.10.0 installed via Homebrew

Comment

When I flip lines, so that [ ] geojson property isn't first, the code doesn't fail.

{ "type": "FeatureCollection", "features": [ {"type": "Feature", "properties": { "a": ""}, "geometry": { "type": "Point", "coordinates": [ [-74, 5]] } }, {"type": "Feature", "properties": { "a": [ ]}, "geometry": { "type": "Point", "coordinates": [ [-74, 5]] } } ] }

Fix

I propose to fall back to raw string value if json field property isn't decodable. When we look at different json values allowed by RFC, and how it's handled by json.loads() the string is the only one that requires different handling.

json.loads("1.1") returns float json.loads("1") returns int json.loads("[]") returns list json.loads("{}") returns dict json.loads("null") returns None json.loads("a") raises exception. Extra quotes required 

When geojson is processed, at the point of calling JSONField.get() we know that geojson is a valid json so the above cases are the only possible ones. json.loads("[") is impossible, so the decoding exception is possible only in case of the string.

[sgillies]

@jbylina Thanks for the suggestion!

How about a log message to say that we are defaulting to different behavior? And a warning so that users can raise an exception instead if they want to be very strict.

[sgillies]

Suggested change

	except json.JSONDecodeError:
	return val
	except json.JSONDecodeError as error:
	warnings.warn(f"JSON field value not decoded, returning as string: {val=}, {error=}", FeatureWarning)
	log.info("JSON field value not decoded, returning as string: val=%r, error=%r", val, error)
	return val