📋 Basic Information

• Service Providers (SPs): Node 4, Node 5
• Current Status: [x] Approved | [ ] Pending
• Reviewer: @beck-8
• Review Started: 2026-03-19 15:30 UTC

⚠️ Failing Metrics

• Storage data indicators dropped severely within 12 hours
• IPNI Verification Failed:
  • Error Message: root CID not verified
  • Affected Deals:
    • Node 4: e4d0b883-b943-48de-a39b-e6ca4a62baa6
    • Node 5: 744e268f-a33b-4a4e-8a0f-6ea6ae5bd873
  • Stack Trace: IpniAddonStrategy.startIpniMonitoring (line 176). The system is failing to verify the root CID on the IPNI indexer after the upload completion.

🔍 Investigation Details

• Within Maintenance Window? [ ] Yes | [x] No (Outside 07:00 and 22:00 UTC windows)
• Investigation Window Deadline (ETL): 2026-03-19 19:30 UTC
• Initial Assessment:
  • Transient: Monitoring for self-recovery.
  • Ongoing Issue: Both nodes are hitting the same verification error simultaneously.
  • Tooling/Dealbot Issue: Investigation required to see if IpniAddonStrategy logic is overly restrictive or if the IPNI API is rate-limiting our requests.

💬 Communication Log

• Channels: [ ] #pdp-endorsed-sp | [x] #fil-pdp-mainnet-launch | [ ] Direct Message
• Status: [ ] Notification Sent | [x] Pending Notification
• SP Response: (To be updated)

🚀 Final Outcome

• Maintain Approval: Issue resolved within the 4-hour window.
• Remove Approval: Thresholds not met by the deadline or critical failure confirmed.

📝 Raw Logs

{"type":"error","name":"Error","message":"IPNI verification failed for deal e4d0b883-b943-48de-a39b-e6ca4a62baa6: root CID not verified","stack":"Error: IPNI verification failed for deal e4d0b883-b943-48de-a39b-e6ca4a62baa6: root CID not verified\n at IpniAddonStrategy.startIpniMonitoring (file:///app/apps/backend/dist/deal-addons/strategies/ipni.strategy.js:176:23)\n at process.processTicksAndRejections (node:internal/process/task_queues:104:5)\n at async IpniAddonStrategy.onUploadComplete (file:///app/apps/backend/dist/deal-addons/strategies/ipni.strategy.js:122:9)\n at async Promise.all (index 0)"}