API Specifications Onboarding Best Practice
Here at FixSpec, we often describe our software using the word "onboarding".
We talk about how we automate the low-value and error-prone, manual tasks in onboarding. We talk about how we onboarding, and how we make onboarding more systematic, streamlined and rigorous.
We talk too about how this efficiency is underpinned by what we see as the inevitable and necessary move towards electronic specifications, an idea supported by an increasing majority of people's guts, but which - at a practical level - seems elusive, non-urgent and low-value.
Three recent, unrelated conversations have made me re-evaluate this outlook, however, and search for a new theory and vocabulary for our company goals.
The first conversation started off as an innocent question from a friend who until recently worked at one of the world's largest global investment banks. Noting the buy-side trend of reducing their broker lists, he innocently questioned whether there really was much "onboarding" going on nowadays (outside of "newly electronic" asset classes or start-ups) to justify buying tools to automate it.
The second observation came during a catch-up chat with a long-term contact at a large European investment bank. After I described recent improvements to our documentation editing toolset and he described how his firm is exposing more granular algo controls to clients, he questioned the return on time invested in documentation tooling when their specification "hadn't changed in donkey's years".
The third remark came during a sales visit to a software vendor seeking to rapidly grow their customer base. While the conversation started off as a discussion about how we can help them cope with an anticipated surge in customers, the follow-up emails instead prioritised internal documentation. Of particular focus was the tracking of translations between inbound and outbound APIs.
So how are these related, and what reasonable conclusions can we draw from these varied experiences?
Let's start with whether there really is a large market for new-customer onboarding. While you may not expect a leading supplier of "onboarding" tooling to admit it, the brutally honest answer is probably no... at least if we limit our definition of "onboarding" to new customers only. So why do firms invest in them?
Take for example one of our customers - one of the world's largest exchange groups. Despite their historic position, deep liquidity and broad product offering, even they do not see large spikes in new customers; their primary use case is about continually re-certifying their existing customer base as functional upgrades and releases arrive. It's about managing the business-as-usual task of maintaining the highest levels of compliance and oversight. Automation makes functional upgrades smoother and faster for both them and their members.
The situation is different for banks and vendors, however, for two main reasons. Firstly, many of these firms pride themselves on the fact that - in the name of customer service - they are able to flex the API exposed to prospects, effectively making their specification a template for discussion as opposed to a fixed agreement. Secondly, these firms typically act as a middle-man, sending the received orders (or fragments of them) outbound to other brokers or exchanges, and so their primary task is to "glue" together inbound and outbound APIs and manage the mappings between them.
When combined, these two factors mean that banks and vendors are internalising the pain of tracking and managing these API differences over time, a problem which grows exponentially as the number of inbound/outbound permutations increases, and as the number and complexity of overrides increases.
It is this de-coupling of inbound and outbound APIs that allowed my second contact to claim that their customer-facing specifications "haven't changed for donkey's years". It's true... but it's only true because the bank's developers and "onboarding" teams are peddling like crazy behind the scenes to prevent it from changing! The bank is still continually changing their internal systems and still upgrading outbound APIs. It is just that customers are shielded from these changes by a policy of full backwards compatibility towards the client. Or expressed differently - it is not that there is no work to do around documentation and testing, merely that the customer doesn't see it.
SIDE NOTE: One point my friend brought up is the fact that MiFID 2 now requires firms to do more rigorous testing around algo changes, including the need to re-certify changes with clients. This is true, but the above analysis suggests a glaring loophole in the regulation; if the bank makes their client API fully backwards-compatible, then there's nothing for the client to re-certify!
The practice of flexing an API for customers seems great for sales guys rushing to start earning commission from a new client, but in software development, these overrides are what is often called "technical debt". Unless the debt is paid back (removed) over time, they act as a dead-weight on firms future API agility. Just ask someone working in a large investment bank or vendor how hard juggling - or even understanding - these overrides and transformations is, and you quickly understand (a) why they never have time to meet with you, and (b) why an all-client re-certification project takes about 6 months to complete.
This is actually the real "onboarding" and documentation problem pain-point today, as raised in my third sales call. In all of my travels, I've only ever heard the same story being told; API overrides are agreed with clients over the phone or email, they are captured in code or highly-technical configurations which only very technical staff to understand, and that the context of why an override exists is lost over time as staff churns. The knock-on impact of a given internal change become increasingly hard to predict or understand, which in turn encourages greater use of custom fields/mappings/hard-codes, thus amplifying the problem for the next iteration.
Personally, I see this combination of (1) legacy API overrides, (2) a general absence of internal controls around how API transformations are agreed and stored, and (3) a policy of not regularly re-certifying customers, as a perfect storm of operational risk. Today, the risk is managed by simply compensating customers for errors when they occur, but I see this as a ticking compliance time-bomb waiting to go off. It's only a matter of time before regulators wake up and take action - MiFID 3 perhaps?
I started this article though by suggesting I should change my perspective and vocabulary, and so I should conclude with my three biggest takeaways from this light-bulb moment.
Firstly, the word "onboarding" is often narrowly interpreted as new client acquisition only and is, therefore, something I plan to strike from my vocabulary. Yes, Central does a great job for new customers, but it is also for maintaining and upgrading customers over time.
Secondly, the idea of "documentation" isn't limited to the public-facing PDF documents most people typically think of. That is - in fact - only one small part of what SpecServer does today. By supporting customer-specific specifications and aggregating multiple documents into a firm-wide repository, SpecServer is already capable of acting as a central store for all things API, and of delivering the transparency, management and audit-trail to start becoming more joined up and understanding the ripple impact of an API change across the firm.
And last but not least, I recognise that the problems experienced by banks and vendors today are cross-functional in nature (and not related to "onboarding" teams only), and therefore require cross-functional solutions. In particular, API overrides agreed with an "onboarding" team become important inputs for product managers evolving products, for development and QA teams working on both internal systems and external APIs, and for business owners seeking answers as to why clients either take so long to start trading, or why they don't have access to certain features or functions.
One of the biggest challenges facing vendors such as FixSpec in this space is how to boost efficiency for our customers while respecting their legacy processes and toolkits; every implementation is different. But while everybody starts from a different place, I believe everybody's destination is the same - faster, smoother and more-joined up API connectivity. Solving that problem takes a radical, enterprise-wide approach.