I am working on a map to visualize river network data using ArcGIS SDK for JavaScript. The data I’m using is from this ArcGIS feature service published by the CLMS. The data includes a dozen feature layers, each corresponding to a Strahler number of the rivers systems, with fields such as OBJECT_ID, upstream and downstream nodes IDs (and more).
Objective: When a river is clicked, I want to highlight all upstream rivers on the map to visualize the patterns formed by all streams of a common drainage basin.
Approach:
- I use JavaScript code with the ArcGIS SDK to display the data on a map and highlight features when clicked.
- My code involves querying the upstream features based on the clicked feature and its upstream node, and iterating the query to reach all upstream features of the river system.
(EDIT: I have added below my question a complete minimal code example as asked in comments, instead of just a short snippet showing the heart of the approach).
Performance issue: The current approach is functioning, but for large rivers, it is very slow. When clicking near the mouth of a ~50 km river, it takes about 20 seconds to find all features to highlight, but when clicking a ~300 km river it takes about 15 minutes. The queries are performed on LayerViews (not the full Layers) to optimize the process, which limits the scope to visible features, but performance remains an issue.
Question: Is there a way to speed up the process of querying upstream features?
I believe the performance bottleneck might be the repeated query inside a while loop, but I can’t find a way to avoid it. Since flow direction is important, I can’t only use a geospatial query (for example querying all adjacent features), as I need to take features attributes (up- and downstream nodes) into account.
I am thinking of maybe writing a small backend service in python, which I am more familiar with, to handle these queries, and possibly of creating a graph structure for the data. But that would involve downloading and pre-processing the whole dataset.
Code
Here is the full functioning code as requested in comments (stripped of a couple of unnecessary things, to obtain a complete minimal reproducible example). For the moment this requires an ArcGIS API key to work.
import esriConfig from '@arcgis/core/config.js'; import Map from '@arcgis/core/Map.js'; import MapView from '@arcgis/core/views/MapView.js'; import FeatureLayer from '@arcgis/core/layers/FeatureLayer.js'; import Handles from '@arcgis/core/core/Handles.js'; esriConfig.apiKey = "XXX"; const LAYERS_IDS = ['5', '6', '7', '8', '9', '10', '11', '12', '13', '15', '16', '17', '18']; const SERVICE_URL = 'https://image.discomap.eea.europa.eu/arcgis/rest/services/EUHydro/EUHydro_RiverNetworkDatabase/MapServer'; // eslint-disable-line max-len -- URL is too long // Create Map and MapView const map = new Map({basemap: 'arcgis/topographic'}); const view = new MapView({ map: map, center: [-1.5, 43.5], zoom: 8, container: 'mapDiv', }); // Add layers to Map const riverLayers = []; // Initiate array of river layers for access in click event const riverLayerViews = []; LAYERS_IDS.forEach((layerId) => { const url = `${SERVICE_URL}/${layerId}`; const riverLayer = new FeatureLayer({ url: url, outFields: ['CatchID', 'FNODE', 'nameText', 'OBJECT_ID', 'TNODE'], }); map.add(riverLayer); riverLayers.push({id: layerId, layer: riverLayer}); view.whenLayerView(riverLayer).then((layerView) => { riverLayerViews.push({id: layerId, layerView: layerView}); }); }); // Declare variables to handle features highlighting let clickedFeatureId = ''; const highlightHandles = new Handles(); // Define click event to interact with the riverLayers view.on('immediate-click', (event) => { // Restrict click events to riverLayers const opts = {include: riverLayers.map((riverLayer) => riverLayer.layer)}; view.hitTest(event, opts).then( async (response) => { if (response.results.length) { // Check if a feature was clicked const feature = response.results[0].graphic; const objectId = feature.attributes.OBJECT_ID; if (clickedFeatureId != objectId) { // Check if new feature was clicked // Read feature information const layerId = feature.layer.layerId.toString(); const upstreamNode = feature.attributes.FNODE; clickedFeatureId = objectId; // Remove previous highlight if (highlightHandles.has()) { highlightHandles.removeAll(); } // Find all upstream features const upstreamFeatures = await queryUpstreamFeatures( layerId, objectId, upstreamNode, riverLayerViews, ); // Highlight upstream features highlightFeatures(upstreamFeatures, riverLayerViews, highlightHandles); } } else { // If no feature was clicked, remove highlight highlightHandles.removeAll(); clickedFeatureId = ''; } }, ); }); /** Query upstream features. The list of features returned includes the feature given as input. */ async function queryUpstreamFeatures(layerId, objectId, baseNode, riverLayerViews) { const upstreamFeatures = [{layerId: layerId, objectId: objectId}]; let nodes = [baseNode]; while (nodes.length > 0) { const newNodes = []; const promises = []; nodes.forEach((node) => { riverLayerViews.forEach((riverLayerView) => { const query = riverLayerView.layerView.createQuery(); query.where = `TNODE = '${node}'`; const promise = riverLayerView.layerView.queryFeatures(query).then((results)=>{ results.features.forEach((feature)=>{ const attributes = feature.attributes; upstreamFeatures.push({layerId: riverLayerView.id, objectId: attributes.OBJECT_ID}); newNodes.push(attributes.FNODE); }); }); promises.push(promise); }); }); await Promise.all(promises); nodes = newNodes; } return upstreamFeatures; } /** Highlight a list of features. */ async function highlightFeatures(upstreamFeatures, riverLayerViews, highlightHandles) { upstreamFeatures.map(async (feature) =>{ const riverLayerView = riverLayerViews.find( (layerView) => layerView.id === feature.layerId, ).layerView; const query = riverLayerView.createQuery(); query.where = `OBJECT_ID = '${feature.objectId}'`; const ids = await riverLayerView.queryObjectIds(query); highlightHandles.add(riverLayerView.highlight(ids)); }); } 
riverLayerViewsandhighlightFeaturesto the question.queryUpstreamFeaturesandhighlightFeaturesare called whenview.stationary === true). I did not include in my question in order to give a minimal example of the performance issue. As a side note, I currently intend to apply the solutions suggested by @Hornbydd, of investigating trace network, or building a graph structure. I just haven't taken the time to do so yet. But if you find another solution it would also be welcome.