A vendor demo is a performance. The vendor chooses the data, the portfolio, the scenario, and the moment at which everything works. Buyers sit through three or four of these, form an impression, and then try to turn that impression into a decision they can defend to a board.
It rarely holds up. Six months later nobody can explain why one platform was chosen over another, and the requirements that mattered most were never tested at all.
One North American independent power producer (IPP) took a different route. It engaged capSpire to run a structured risk and credit vendor evaluation to complement its existing ETRM system, with every option assessed on identical terms, including extending the capability already in place.
When a trading book diversifies faster than its analytics
The business trades natural gas, power, renewable energy certificates and carbon offsets, and financial transmission rights, across all North American markets. It is also expanding into battery energy storage.
That last point changes the analytical requirement more than it sounds. Battery assets bring dispatch optimization, degradation modeling, and independent system operator market data into a risk function that was already covering four commodity types across all the markets.
At the same time, the standard for institutional risk management has moved. The risk team wanted the ability to backtest value-at-risk predictions against realized profit and loss, to backcast stochastic price distributions, and to govern model calibration with a documented audit trail. Those are expectations that barely existed in most trading businesses a decade ago and are now the baseline for a risk function of this scale.
So the requirement set had grown in three directions at once: more commodities, a new asset class with its own physics, and a higher bar for model validation. Any organization in that position reaches a point where it has to decide deliberately where each capability should live.
Where advanced risk analytics belongs in the technology stack
This is the question the assessment was really built to answer.
A trading and risk platform is the book of record. Its job is to capture the trade, hold the position, and control the process around it, and the platform in place does that job. Advanced quantitative analytics is a different discipline: value-at-risk across a cross-commodity book, stress testing and scenario analysis, stochastic model validation, and battery dispatch optimization all sit naturally in a dedicated analytics layer alongside the core system.
That is normal architecture rather than a compromise. The specialist analytics market exists precisely because these models evolve faster than any core platform release cycle, and because the people who build them are quantitative analysts rather than trading system architects.
The company was already evaluating analytics platforms when capSpire was engaged. The open questions were which one fit the portfolio, and how the analytics layer and the book of record would exchange data in a controlled and reconcilable way. Without that second piece, prices and risk figures produced in the analytics layer cannot flow back into the system of record cleanly, and the middle and back office end up reconciling by hand.
Two other requirements sat alongside it. The company wanted governed business intelligence reporting tied to live trading data, with standard templates aligned to trading, risk, and operations measures, rather than the manual reporting its teams were producing each day. In addition, it wanted the data pipelines underneath that reporting built properly.
What an evidence-based ETRM vendor evaluation looks like
capSpire ran a structured assessment over two to three months, with a team combining risk and credit subject matter expertise, consulting delivery, and technology architecture, working directly with the head of risk and the chief technology officer.
The work began with a current state review rather than a vendor conversation. Existing capability was mapped against future-state requirements using capSpire’s benchmark matrix and software requirements list, and requirements were then consolidated across three areas: risk measurement, credit management, and regulatory compliance.
That order matters. A requirements list assembled after the demos tends to describe what the platforms happened to show. The same list assembled before them describes what the business actually needs.
Six scenarios built to make the platforms prove it
Rather than accept vendor led demonstrations, capSpire wrote six scenario scripts and required every candidate to run them. Each script specified the portfolio composition, the market scenario, and the output the system was expected to produce.
The six scenarios covered:
- Portfolio value-at-risk and stress testing under a natural gas price shock, using regional swaps, power forwards, and options
- Battery asset valuation and dispatch optimization for a 4-hour lithium ion system in real time and day ahead markets
- Environmental attribute option valuation across two regional environmental markets
- Financial transmission rights strategy and congestion risk under a 60% congestion price increase
- Value-at-risk backtesting over a one year rolling historical window, including exception counts and breach analysis
- Stochastic model validation and backcasting covering Monte Carlo and jump diffusion methods, quantile plots, error measurement, and recalibration
Each platform was then scored on a one to five scale against every scenario, and separately against general capability dimensions including data management, application programming interface integration, dashboarding, workflow governance, scalability, and model library breadth.
The result is a comparison that survives scrutiny, because the same portfolio and the same market shock were put to every platform.
Why the client made the decision, not the advisor
Two platforms advanced to the final round and were evaluated against the defined scenarios and capability areas. The supporting scores and rationale were documented in detail.
Two of the six scenarios were not completed during the demonstrations. capSpire identified these as open items requiring further validation before a selection could be made. The capabilities were not scored based on assumptions or treated as proven without evidence.
The assessment resulted in three key deliverables: a requirements summary, a vendor evaluation matrix, and a future-state technology roadmap with phased implementation guidance. Most importantly, the client received a transparent, evidence-based comparison that its team could confidently support and defend internally.
Reporting and integration are part of the same problem
A platform decision on its own does not build a reporting layer, and it does not connect an analytics platform to a book of record. capSpire set out a phased program covering both, to run once the selection was made.
The first strand builds governed business intelligence reporting: interactive trading, risk, and operations dashboards, with workspace governance, access controls, and reliable data pipelines from the trading system into the reporting layer.
The second builds the integration between the selected analytics platform and the core trading system, covering interface connections, automated data feeds, error handling, reconciliation controls, and logging.
The third provides implementation and project governance through the full vendor implementation, from initiation to steady state operation, with knowledge transfer to the internal team as an explicit objective rather than a closing formality. Together the program spans nine to thirteen months from selection through to a stable production environment.
The reasoning behind all three is the same. The company did not have the internal capacity or the specialized expertise to run vendor selection, system implementation, reporting build-out, and integration governance at the same time. What it needed was an independent advocate in the room rather than another vendor.
Let’s talk
If you are deciding where a capability should sit, facing a vendor decision, or running reporting by hand that should be automated, reach out to capSpire’s Advisory team.
Kayne Coulter
Managing Director, Advisory – North America
