Skip to main content

Want To Use Data? Get The Uncut Stuff

If you can use Excel, you can be a quant.
  • Author:
  • Updated:

In case anyone had any doubts that the world of finance is changing, we need look no further than the recent SEC announcement that it was investigating stock manipulation through articles posted to investment website SeekingAlpha. That case highlights an important reality in the financial business – increasingly, areas that were once seen as the purview of humans like reading and analyzing information about a firm, can now be done effectively by computers.


I know SeekingAlpha very well – I used to be a staff member there. Specifically I worked on the PRO product, and among other things I worked on Big Data projects for them to help turn their content into useable data for institutional investors. SeekingAlpha is trying to disrupt the traditional model of analyst coverage in finance, but the truth is that analysts themselves are being disrupted by advances in data analytics.

Finance professionals often point to what they see as the subjective nature of their work and the difficulty that exists in translating those concepts into data. The reality is that people who think this way are missing a profound shift in the industry – a shift towards what is called unstructured data.

Unstructured data simply means any type of data that does not fit neatly into a spreadsheet. The vast majority of communication fits that model. Yet unstructured data from phone call transcripts to court opinions can all be transformed into useable data – and it’s not hard to do. In fact, anyone with Excel and a transcript can begin doing data analytics on their own.

Try this exercise for fun sometime – take this article, remove all punctuation, then import it into excel using space delimiters. When you finish you will have a list of every word used here. From there, use Excel’s count function to tally the frequency of each word, and use a word list from one of the research portals on the internet to decide which words are positive and negative. Once you have done that, you will have a measure of the “sentiment” in this article. You can repeat the process for any text document. Sentiment is just the tip of the iceberg in terms of what can be measured with these techniques.

That type of transformation of text data into a sentiment score has been used in many applications, from predicting stock market returns based on “crowd wisdom” to analyzing executive commentary on conference calls. Moreover, the data analytics work that is being produced is not coming from firms selling data analytics packages but independent researchers, large asset managers, and institutional investors.

The commentary that analysts produce was once considered subjective and impossible to measure – it is not. Some parts of the financial industry are starting to come around to this realization. I am currently working on a project with a large asset manager to use unstructured data to identify the “value” of a trip made by a sales guy or a conference sponsored by the asset manager with speaking engagements by the partners. This type of analysis is a key part of helping the finance industry become more effective.

All of this is an upfront investment, of course, but once that investment is made it’s easy to examine massive amounts of unstructured data and make decisions accordingly.

More broadly, the point for professionals of all stripes is that even if a decision seems to be inherently subjective, it is likely that data play a role behind the scenes. Figuring out how to take the mass of information and turn it into insight is tricky, but not impossible.

Does this mean that financial advisors are outdated or that professions like the finance are doomed? Of course not.

What it does mean is that FAs need to look at how the insights from non-traditional data analytics can complement their own skill set. Finance has already seen this change occurring. There are fewer traditional traders in finance at this point. Instead, former traders often work with quants to develop automated algorithms that enable trading.

The finance profession is changing, and hiding behind a veneer of subjectivity in decision making won’t stop the process. The best financial professionals of tomorrow will be the ones that capitalize on that change today.



Data Analytics: Garbage In, Garbage Out

Happy New Year, let's talk garbage.


Private Equity Has An Alpha Problem

Do you ever look at the Private Equity space and think "Where did all the alpha go?"


Stocks Can Get It Up Just Fine Without The Fed, Thank You Very Much

The stimulative effects on the stock market of abnormally low interest rates may be greatly exaggerated.


The Struggle To Optimize Fee-Only Models Is Real

Umm, you wanna get in on this logit regression?


Living With Price Shock: A (Maybe) Practical Guide

Can you live with volatility? Can you reeeeeally?

Hedge Funds Are Thirsty As Hell For Decision-Making Data

Fintech startups are making serious bank selling proprietary data to the buyside.


Avoid The HiPPo!

The HiPPo is hungry...for your failure,