A Bernstein analyst recently put the world’s leading generic AI models to the test, asking them to perform like a real financial analyst. (See FT article here). The results were clear: ChatGPT, Gemini, Grok, Claude and their peers all failed to produce reliable models, insights or company reports. They could extract data and make charts. But they couldn’t think like analysts.
Asking generic LLMs to operate like an Analyst is like expecting a secondary school kid to behave like a university finance graduate. The former can have a general understanding of business, but the latter should be able to analyse a company or sector, build a financial model and write a draft report. This is the difference between generic AI and Credit Specialised AI.
Large Language Models (LLMs) like ChatGPT are fundamentally next-word text predictors trained on generic internet content. They’re brilliant at pattern recognition, summarising documents, and answering general questions.
But ask them to build a financial model, assign a rating, or justify a recommendation and things fall apart. Bernstein’s test showed that even with extensive prompts, high-quality data and detailed sector background, the models returned hallucinated numbers and broken spreadsheets.
Why? Because they aren’t trained on industry specific data and methodologies. They don’t have the rules, logic, and domain knowledge that underpin financial research.
“There are too many accounting nuances and country differences,” noted Bernstein’s Lead Analyst. Humans understand this. AI doesn’t, yet.
Below is an exhibit outlining three common approaches to prompting the LLM which fail to deliver humanlike Analyst outcomes.
Generic AI (non-Agentic AI)
A Credit Specialised AI can mimic an Analyst’s thought process and create first draft outcomes for Analyst reviews of the same level as a finance major university graduate.
The specialised AI Agent can extract data from multiple sources across all formats, synthesise everything into positive/neutral/negative sentiment, listing out concerns that may impact credit risk and rating strength like management actions.
Early Warning Signals
Using heterogeneous data and adding Analyst experience to long-term company performance analysis is core to an Analyst's role. Generic AI absolutely fails in creating meaningful models. However, Credit Specialised AI successfully generates meaningful and highly accurate models in spreadsheet format — explaining underpinning calculations and connecting to sources — effectively analysing the outlook for companies catering for accounting norms, managing for country-to-country differences and tackling different languages. This Specialised AI is built on credit domain centric learning to ensure it understands all the subtleties.
Financial Modelling
Forecast and Scenario Analysis
Analysing the outlook for companies and developing a detailed credit research report is again core to an Analyst’s role and something generic AI fails to solve. The Credit Specialised AI handles this task very effectively with in-depth reasoning and connectivity to source to ensure trust. It creates outcomes in-line with the Analyst’s expected modelling template, rules, logic and intent. It also tackles the hallucination problem by verifying all data sources and providing full traceability and auditability to the source.
First Draft Credit Memorandum. Please See the Full Draft Here.
Credit Specialised AI unlocks the full value of Analyst’s expertise with trust at its core.
Building a Credit Specialised AI takes time and requires three key ingredients:
Specialised “Credit-brain” including credit content, knowledge, data and usage
Specialised Multi-Agentic architecture to train and fine-tune the models with a feedback loop
Specialised AI DataFactory combining algorithms with human supervision
A Specialised “Credit-brain” is the context in which the multi-agentic system works. It comprises different layers to handle raw data, extract credit knowledge from the raw data, harmonise and validate the data and finally populate data models for different companies or countries. Think modules of a graduate degree.
Specialised “Credit-Brain”
A Specialised Multi-Agentic architecture trains and fine-tunes models with a feedback loop. The system solves credit problems by applying a group of sub-AI Agents performing specialised tasks covering the data and credit lifecycle. For example, dedicated Ingestion Agents are trained to perform tasks related to ingestion of structured, unstructured and semi-structured data. Metadata Agents are used to tag data connecting data with knowledge e.g., the definition and computation of EBITDA. An Ontology Agent is used to embed credit rules and knowledge for a single company or a sector e.g. accounting rules, country by country differentiators by company. The Analytics Agent builds a financial model bringing together all the key financial metrics and ratios in line with defined rules and logic and with full traceability and connectivity to the source document or raw data. The Summary Agent composes a credit research report with in-depth reasoning providing a deep dive into why the numbers are moving and their implications for credit risk. Think of these agents representing different domains of expertise acquired via experience on the job.
Each of these highly specialised agents must be able to translate intent (expressed in natural language) into an execution plan, introspect and course correct during task execution, invoke tools, access persistent memory and enrich own context with relevant data. Critically, the agents must be able to learn and improve from human feedback.
Credit Specialised AI (Agentic AI)
A Specialised AI DataFactory ensures trust, quality and relevance of insights generated by the Credit Agent. It supports the effective running of the Agents by combining algorithms with human feedback. The algorithms perform a Detect, Correct and Repair approach which is reenforced by expert oversight. Detection entails algorithmic recognition of errors; Correction entails automated fixing of errors; and Repair entails elevation of humans to supervisory roles, overseeing outcomes of AI agents and intervening via code to fix data issues arising from algorithmic processing. Human supervisors guide each stage of the process, enhancing the quality of company insights. Think graduate supervisor or manager.
Human Supervised Algorithmic Data Processing
The role of the analyst isn’t going away, it’s being upgraded.
With Credit Specialised AI doing the heavy lifting and delivering a foundation, analysts can focus efforts where they add their unique value and react more quickly than peers to credit events. They can truly deliver a risk-based approach with deeper analysis and improved decision-making on higher risk and more complex counterparts. They become a real first line of defence rather than spreadsheet processors and administrators.
The most important limitation of generic or specialised AI is the risk of over reliance. It is a way for Analysts to do their jobs better, but with a continued focus on individual accountability and using it as a tool to enhance what they are doing. Not to replace what they are doing.
And it’s not just the experienced Analysts who benefit. The technology will really improve the quality of the work of junior Analysts. They are going to get high-impact work while learning from the Specialised AI or getting it to other things.
One thing is for sure “If I was an analyst today, I would be using this tool day-in-day-out to make me smarter and deliver high-impact outcomes faster.”
Generic AI can’t write good Analyst research. It can’t model. It can’t reason. And it definitely can’t be trusted with risk.
Credit Specialised AI can.
Specialised AI delivers on the real promise of trustworthy automation, domain-specific insight and radical productivity gains.
“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” is a quote from Mark Twain.
And Generic AI knows for sure what just ain’t so — and that is a big problem for Analysts — the solution to this problem is Specialised AI.
This is not a future vision. It’s LIVE. In 30 plus global financial institutions.