feat: Implement ST_LENGTH geography function (#1791)

* feat: Implement ST_LENGTH geography function This commit introduces the ST_LENGTH function for BigQuery DataFrames. ST_LENGTH computes the length of GEOGRAPHY objects in meters. The implementation includes: - A new operation `geo_st_length_op` in `bigframes.operations.geo_ops`. - The user-facing function `st_length` in `bigframes.bigquery._operations.geo`. - Exposure of the new operation and function in relevant `__init__.py` files. - Comprehensive unit tests covering various geometry types (Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection), empty geographies, and NULL inputs. The function behaves as per the BigQuery ST_LENGTH documentation: - Returns 0 for POINT, MULTIPOINT, and empty GEOGRAPHYs. - Returns the perimeter for POLYGON and MULTIPOLYGON. - Returns the total length for LINESTRING and MULTILINESTRING. - For GEOMETRYCOLLECTION, sums the lengths/perimeters of its constituent linestrings and polygons. * feat: Add NotImplemented length property to GeoSeries This commit adds a `length` property to the `GeoSeries` class. Accessing this property will raise a `NotImplementedError`, guiding you to utilize the `bigframes.bigquery.st_length()` function instead. This change includes: - The `length` property in `bigframes/geopandas/geoseries.py`. - A unit test in `tests/system/small/geopandas/test_geoseries.py` to verify that the correct error is raised with the specified message when `GeoSeries.length` is accessed. * Update bigframes/bigquery/_operations/__init__.py * fix lint * add missing compilation method * use pandas for the expected values in tests * fix: Apply patch for ST_LENGTH and related test updates This commit applies a user-provided patch that includes: - Removing `st_length` from `bigframes/bigquery/_operations/__init__.py`. - Adding an Ibis implementation for `geo_st_length_op` in `bigframes/core/compile/scalar_op_compiler.py`. - Modifying `KMeans` in `bigframes/ml/cluster.py` to handle `init="k-means++"`. - Updating geo tests in `tests/system/small/bigquery/test_geo.py` to use `to_pandas()` and `pd.testing.assert_series_equal`. Note: System tests requiring Google Cloud authentication were not executed due to limitations in my current environment. * feat: Add use_spheroid parameter to ST_LENGTH and update docs This commit introduces the `use_spheroid` parameter to the `ST_LENGTH` geography function, aligning it more closely with the BigQuery ST_LENGTH(geography_expression[, use_spheroid]) signature. Key changes: - `bigframes.operations.geo_ops.GeoStLengthOp` is now a dataclass that accepts `use_spheroid` (defaulting to `False`). A check is included to raise `NotImplementedError` if `use_spheroid` is `True`, as this is the current limitation in BigQuery. - The Ibis compiler implementation for `geo_st_length_op` in `bigframes.core.compile.scalar_op_compiler.py` has been updated to accept the new `GeoStLengthOp` operator type. - The user-facing `st_length` function in `bigframes.bigquery._operations.geo.py` now includes the `use_spheroid` keyword argument. - The docstring for `st_length` has been updated to match the official BigQuery documentation, clarifying that only lines contribute to the length (points and polygons result in 0 length), and detailing the `use_spheroid` parameter. Examples have been updated accordingly. - Tests in `tests/system/small/bigquery/test_geo.py` have been updated to: - Reflect the correct behavior (0 length for polygons/points). - Test calls with both default `use_spheroid` and explicit `use_spheroid=False`. - Verify that `use_spheroid=True` raises a `NotImplementedError`. Note: System tests requiring Google Cloud authentication were not re-executed for this specific commit due to environment limitations identified in previous steps. The changes primarily affect the operator definition, function signature, and client-side validation, with the core Ibis compilation logic for length remaining unchanged. * feat: Implement use_spheroid for ST_LENGTH via Ibis UDF This commit refactors the ST_LENGTH implementation to correctly pass the `use_spheroid` parameter to BigQuery by using Ibis's `ibis_udf.scalar.builtin('ST_LENGTH', ...)` function. Key changes: - `bigframes.operations.geo_ops.GeoStLengthOp`: The client-side `NotImplementedError` for `use_spheroid=True` (raised in `__post_init__`) has been removed. BigQuery DataFrames will now pass this parameter directly to BigQuery. - `bigframes.core.compile.scalar_op_compiler.geo_length_op_impl`: The implementation now always uses `ibis_udf.scalar.builtin('ST_LENGTH', x, op.use_spheroid)` instead of `x.length()`. This ensures the `use_spheroid` parameter is included in the SQL generated for BigQuery. - `tests/system/small/bigquery/test_geo.py`: - The test expecting a client-side `NotImplementedError` for `use_spheroid=True` has been removed. - A new test `test_st_length_use_spheroid_true_errors_from_bq` has been added. This test calls `st_length` with `use_spheroid=True` and asserts that an exception is raised from BigQuery, as BigQuery itself currently only supports `use_spheroid=False` for the `ST_LENGTH` function. - Existing tests for `st_length` were already updated in a previous commit to reflect that only line geometries contribute to the length, and these continue to verify behavior with `use_spheroid=False`. This change ensures that BigQuery DataFrames accurately reflects BigQuery's `ST_LENGTH` capabilities concerning the `use_spheroid` parameter. * refactor: Use Ibis UDF for ST_LENGTH BigQuery builtin This commit refactors the ST_LENGTH geography operation to use an Ibis UDF defined via `@ibis_udf.scalar.builtin`. This aligns with the pattern exemplified by other built-in functions like ST_DISTANCE when a direct Ibis method with all necessary parameters is not available. Key changes: - A new `st_length` function is defined in `bigframes/core/compile/scalar_op_compiler.py` using `@ibis_udf.scalar.builtin`. This UDF maps to BigQuery's `ST_LENGTH(geography, use_spheroid)` function. - The `geo_length_op_impl` in the same file now calls this `st_length` Ibis UDF, replacing the previous use of `op_typing.ibis_function`. - The `GeoStLengthOp` in `bigframes/operations/geo_ops.py` and the user-facing `st_length` function in `bigframes/bigquery/_operations/geo.py` remain unchanged from the previous version, as they correctly define the operation's interface and parameters. This change provides a cleaner and more direct way to map the BigQuery DataFrames operation to the specific BigQuery ST_LENGTH SQL function signature, while maintaining the existing BigQuery DataFrames operation structure. The behavior of the `st_length` function, including its handling of the `use_spheroid` parameter and error conditions from BigQuery, remains the same. * refactor: Consolidate st_length tests in test_geo.py This commit refactors the system tests for the `st_length` geography function in `tests/system/small/bigquery/test_geo.py`. The numerous individual test cases for different geometry types have been combined into a single, comprehensive test function `test_st_length_various_geometries`. This new test uses a single GeoSeries with a variety of inputs (Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection, None/Empty) and compares the output of `st_length` (with both default and explicit `use_spheroid=False`) against a pandas Series of expected lengths. This consolidation improves the conciseness and maintainability of the tests for `st_length`. The test for `use_spheroid=True` (expecting an error from BigQuery) remains separate. * fix: Correct export of GeoStLengthOp in operations init This commit fixes an ImportError caused by an incorrect name being used for the ST_LENGTH geography operator in `bigframes/operations/__init__.py`. When `geo_st_length_op` (a variable) was replaced by the dataclass `GeoStLengthOp`, the import and `__all__` list in this `__init__.py` file were not updated. This commit changes the import from `.geo_ops` to correctly import `GeoStLengthOp` and updates the `__all__` list to export `GeoStLengthOp`. * fix system test and some linting * fix lint * fix doctest * fix docstring * Update bigframes/core/compile/scalar_op_compiler.py --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>

Author: tswast

bigframes/bigquery/__init__.py

@@ -32,6 +32,7 @@
  st_difference,
  st_distance,
  st_intersection,
+ st_length,
 )
 from bigframes.bigquery._operations.json import (
  json_extract,
@@ -58,6 +59,7 @@
  "st_difference",
  "st_distance",
  "st_intersection",
+ "st_length",
  # json ops
  "json_extract",
  "json_extract_array",

bigframes/bigquery/_operations/geo.py

@@ -380,3 +380,67 @@ def st_intersection(
  each aligned geometry with other.
  """
  return series._apply_binary_op(other, ops.geo_st_intersection_op)
+
+
+def st_length(
+ series: Union[bigframes.series.Series, bigframes.geopandas.GeoSeries],
+ *,
+ use_spheroid: bool = False,
+) -> bigframes.series.Series:
+ """Returns the total length in meters of the lines in the input GEOGRAPHY.
+
+ If a series element is a point or a polygon, returns zero for that row.
+ If a series element is a collection, returns the length of the lines
+ in the collection; if the collection doesn't contain lines, returns
+ zero.
+
+ The optional use_spheroid parameter determines how this function
+ measures distance. If use_spheroid is FALSE, the function measures
+ distance on the surface of a perfect sphere.
+
+ The use_spheroid parameter currently only supports the value FALSE. The
+ default value of use_spheroid is FALSE. See:
+ https://cloud.google.com/bigquery/docs/reference/standard-sql/geography_functions#st_length
+
+ **Examples:**
+
+ >>> import bigframes.geopandas
+ >>> import bigframes.pandas as bpd
+ >>> import bigframes.bigquery as bbq
+ >>> from shapely.geometry import Polygon, LineString, Point, GeometryCollection
+ >>> bpd.options.display.progress_bar = None
+
+ >>> series = bigframes.geopandas.GeoSeries(
+ ... [
+ ... LineString([(0, 0), (1, 0)]), # Length will be approx 1 degree in meters
+ ... Polygon([(0.0, 0.0), (0.1, 0.1), (0.0, 0.1)]), # Length is 0
+ ... Point(0, 1), # Length is 0
+ ... GeometryCollection([LineString([(0,0),(0,1)]), Point(1,1)]) # Length of LineString only
+ ... ]
+ ... )
+
+ Default behavior (use_spheroid=False):
+
+ >>> result = bbq.st_length(series)
+ >>> result
+ 0 111195.101177
+ 1 0.0
+ 2 0.0
+ 3 111195.101177
+ dtype: Float64
+
+ Args:
+ series (bigframes.series.Series | bigframes.geopandas.GeoSeries):
+ A series containing geography objects.
+ use_spheroid (bool, optional):
+ Determines how this function measures distance.
+ If FALSE (default), measures distance on a perfect sphere.
+ Currently, only FALSE is supported.
+
+ Returns:
+ bigframes.series.Series:
+ Series of floats representing the lengths in meters.
+ """
+ series = series._apply_unary_op(ops.GeoStLengthOp(use_spheroid=use_spheroid))
+ series.name = None
+ return series

bigframes/core/compile/scalar_op_compiler.py

@@ -30,7 +30,6 @@
 import bigframes.core.compile.default_ordering
 import bigframes.core.compile.ibis_types
 import bigframes.core.expression as ex
-import bigframes.dtypes
 import bigframes.operations as ops
 
 _ZERO = typing.cast(ibis_types.NumericValue, ibis_types.literal(0))
@@ -1079,6 +1078,12 @@ def geo_x_op_impl(x: ibis_types.Value):
  return typing.cast(ibis_types.GeoSpatialValue, x).x()
 
 
+@scalar_op_compiler.register_unary_op(ops.GeoStLengthOp, pass_op=True)
+def geo_length_op_impl(x: ibis_types.Value, op: ops.GeoStLengthOp):
+ # Call the st_length UDF defined in this file (or imported)
+ return st_length(x, op.use_spheroid)
+
+
 @scalar_op_compiler.register_unary_op(ops.geo_y_op)
 def geo_y_op_impl(x: ibis_types.Value):
  return typing.cast(ibis_types.GeoSpatialValue, x).y()
@@ -2057,6 +2062,12 @@ def st_distance(a: ibis_dtypes.geography, b: ibis_dtypes.geography, use_spheroid
  """Convert string to geography."""
 
 
+@ibis_udf.scalar.builtin
+def st_length(geog: ibis_dtypes.geography, use_spheroid: bool) -> ibis_dtypes.float: # type: ignore
+ """ST_LENGTH BQ builtin. This body is never executed."""
+ pass
+
+
 @ibis_udf.scalar.builtin
 def unix_micros(a: ibis_dtypes.timestamp) -> int: # type: ignore
  """Convert a timestamp to microseconds"""

bigframes/geopandas/geoseries.py

@@ -30,6 +30,12 @@ def __init__(self, data=None, index=None, **kwargs):
  data=data, index=index, dtype=geopandas.array.GeometryDtype(), **kwargs
  )
 
+ @property
+ def length(self):
+ raise NotImplementedError(
+ "GeoSeries.length is not yet implemented. Please use bigframes.bigquery.st_length(geoseries) instead."
+ )
+
  @property
  def x(self) -> bigframes.series.Series:
  series = self._apply_unary_op(ops.geo_x_op)

bigframes/operations/__init__.py

@@ -101,6 +101,7 @@
  geo_x_op,
  geo_y_op,
  GeoStDistanceOp,
+ GeoStLengthOp,
 )
 from bigframes.operations.json_ops import (
  JSONExtract,
@@ -385,6 +386,7 @@
  "geo_st_geogfromtext_op",
  "geo_st_geogpoint_op",
  "geo_st_intersection_op",
+ "GeoStLengthOp",
  "geo_x_op",
  "geo_y_op",
  "GeoStDistanceOp",

bigframes/operations/geo_ops.py

@@ -80,3 +80,12 @@ class GeoStDistanceOp(base_ops.BinaryOp):
 
  def output_type(self, *input_types: dtypes.ExpressionType) -> dtypes.ExpressionType:
  return dtypes.FLOAT_DTYPE
+
+
+@dataclasses.dataclass(frozen=True)
+class GeoStLengthOp(base_ops.UnaryOp):
+ name = "geo_st_length"
+ use_spheroid: bool = False
+
+ def output_type(self, *input_types: dtypes.ExpressionType) -> dtypes.ExpressionType:
+ return dtypes.FLOAT_DTYPE

tests/system/small/bigquery/test_geo.py

@@ -19,10 +19,14 @@
 from shapely.geometry import ( # type: ignore
  GeometryCollection,
  LineString,
+ MultiLineString,
+ MultiPoint,
+ MultiPolygon,
  Point,
  Polygon,
 )
 
+from bigframes.bigquery import st_length
 import bigframes.bigquery as bbq
 import bigframes.geopandas
 
@@ -59,6 +63,66 @@ def test_geo_st_area():
  )
 
 
+# Expected length for 1 degree of longitude at the equator is approx 111195.079734 meters
+DEG_LNG_EQUATOR_METERS = 111195.07973400292
+
+
+def test_st_length_various_geometries(session):
+ input_geometries = [
+ Point(0, 0),
+ LineString([(0, 0), (1, 0)]),
+ Polygon([(0, 0), (1, 0), (0, 1), (0, 0)]),
+ MultiPoint([Point(0, 0), Point(1, 1)]),
+ MultiLineString([LineString([(0, 0), (1, 0)]), LineString([(0, 0), (0, 1)])]),
+ MultiPolygon(
+ [
+ Polygon([(0, 0), (1, 0), (0, 1), (0, 0)]),
+ Polygon([(2, 2), (3, 2), (2, 3), (2, 2)]),
+ ]
+ ),
+ GeometryCollection([Point(0, 0), LineString([(0, 0), (1, 0)])]),
+ GeometryCollection([]),
+ None, # Represents NULL geography input
+ GeometryCollection([Point(1, 1), Point(2, 2)]),
+ ]
+ geoseries = bigframes.geopandas.GeoSeries(input_geometries, session=session)
+
+ expected_lengths = pd.Series(
+ [
+ 0.0, # Point
+ DEG_LNG_EQUATOR_METERS, # LineString
+ 0.0, # Polygon
+ 0.0, # MultiPoint
+ 2 * DEG_LNG_EQUATOR_METERS, # MultiLineString
+ 0.0, # MultiPolygon
+ DEG_LNG_EQUATOR_METERS, # GeometryCollection (Point + LineString)
+ 0.0, # Empty GeometryCollection
+ pd.NA, # None input for ST_LENGTH(NULL) is NULL
+ 0.0, # GeometryCollection (Point + Point)
+ ],
+ index=pd.Index(range(10), dtype="Int64"),
+ dtype="Float64",
+ )
+
+ # Test default use_spheroid
+ result_default = st_length(geoseries).to_pandas()
+ pd.testing.assert_series_equal(
+ result_default,
+ expected_lengths,
+ rtol=1e-3,
+ atol=1e-3, # For comparisons involving 0.0
+ ) # type: ignore
+
+ # Test explicit use_spheroid=False
+ result_explicit_false = st_length(geoseries, use_spheroid=False).to_pandas()
+ pd.testing.assert_series_equal(
+ result_explicit_false,
+ expected_lengths,
+ rtol=1e-3,
+ atol=1e-3, # For comparisons involving 0.0
+ ) # type: ignore
+
+
 def test_geo_st_difference_with_geometry_objects():
  data1 = [
  Polygon([(0, 0), (10, 0), (10, 10), (0, 0)]),

tests/system/small/geopandas/test_geoseries.py

@@ -96,6 +96,17 @@ def test_geo_area_not_supported():
  bf_series.area
 
 
+def test_geoseries_length_property_not_implemented(session):
+ gs = bigframes.geopandas.GeoSeries([Point(0, 0)], session=session)
+ with pytest.raises(
+ NotImplementedError,
+ match=re.escape(
+ "GeoSeries.length is not yet implemented. Please use bigframes.bigquery.st_length(geoseries) instead."
+ ),
+ ):
+ _ = gs.length
+
+
 def test_geo_distance_not_supported():
  s1 = bigframes.pandas.Series(
  [