Building a data screen/ranking program

[Slava]

As the title says, I want to build (or maybe have someone build?) a program that will gather information and complete calculations. Then based on the outcomes of the calculations it would rank the outcomes. Really what I want is a few steps of this completed to get to a final list, but that's the basic format.

Maybe of no surprise if you know me, In want to do this for stocks. Before anyone says to use a stock screen online somewhere, I can't because the criteria I want to use isn't there (which is what makes this a somewhat proprietary system). So basically what I want to do is have my own system. I know exactly what I want in terms of the formulas, and just need a way to automate it. In all honesty, this doesn't seem like it would be that hard, but I'm not an expert!

So here are my questions:

1. Is this something I can build myself? How hard does this sound?

2. Can anyone suggest any resources for me to check out?

3. Assuming I can't do this myself and its too difficult, what am I looking at to have it done? (I realise this is difficult without knowing the full scope and such, and you can feel free to PM me if you prefer as well).

The outcome doesn't have to look pretty as its purely for my own use.

[photon]

I used to work for a guy and one one of the apps we did was to take stock information and combine that with patent information to try and find an arbitrage. The idea was that often companies would have their patents approved but wouldn't announce it for a few days or weeks, and there seemed to be an advantage there. I think we'd got to the point of examining historical information and finding the premise played out in the big picture.

I think you could build something like this yourself even in just Excel, assuming anyway that you can get a source of data that's appropriate. I would imagine though that there are lots of sources of stock information that could be imported into Excel.

Or are you talking more real-time?

[Slava]

Originally Posted by photon

I used to work for a guy and one one of the apps we did was to take stock information and combine that with patent information to try and find an arbitrage. The idea was that often companies would have their patents approved but wouldn't announce it for a few days or weeks, and there seemed to be an advantage there. I think we'd got to the point of examining historical information and finding the premise played out in the big picture.

I think you could build something like this yourself even in just Excel, assuming anyway that you can get a source of data that's appropriate. I would imagine though that there are lots of sources of stock information that could be imported into Excel.

Or are you talking more real-time?

No, actually it doesn't need to be real-time at all. Maybe the issue is the data that's usable. I know I could construct this to work for say one stock, but its the filtering through thousands that I'm jot sure how to do effectively.

[CaramonLS]

Save some steps and build an HFT machine. Free money.

[photon]

If you can do it for one, then it's just a matter of copying the formula many times isn't it?

Can the formula fit in an Excel spreadsheet formula?

If you had a spreadsheet with Column A being Stock names, Column B being prices, and Column D being the result of the formula, is it that straight forward?

[Slava]

Originally Posted by CaramonLS

Save some steps and build an HFT machine. Free money.

Lol, well its not that "advanced".

Originally Posted by photon

If you can do it for one, then it's just a matter of copying the formula many times isn't it?

Can the formula fit in an Excel spreadsheet formula?

If you had a spreadsheet with Column A being Stock names, Column B being prices, and Column D being the result of the formula, is it that straight forward?

Well its a lot more complex than that. Say I need a specific metric from a set of financials....let's say that I want to take the net revenue of Apple and then divide it into the gross revenue. Now I want to do that for every stock the NYSE, Nasdaq and TSE. (I know that this screen is available...that's not what I'm trying to do here, its an example).

Basically what I want to do is a series of those types of calculations, screen those results. Do more calculations on the remaining results., and then screen those. Then lastly more calculations and rank the remainder.

[photon]

Ah ok, assuming the data is available in an easy to import format (CSV say) it still doesn't sound like anything that Excel couldn't do, if the formulae are very complex then there's even a Visual Basic type language available to write more complex stuff.

[Slava]

So dumb question, can Excel handle like 30,000 calculations, or at least that many stocks? Is my computer going to need a lot of power to run this reasonably?

I'd hate to get all the way through figuring this out and then find out its too big to be handled.

[photon]

I've seen some pretty massive spreadsheets, and I've seen some spreadsheets that took 30 seconds to update, but I don't have enough experience to know where things go bad. 30,000 doesn't sound like very much to me, but I'm more used to developing my own apps.

There's also lots of things you can do with Excel to speed things up as you get more complicated (how things are referenced, using VBA, making sure things are sorted if that's relevant, etc).

I'm not really sure what your next step up would be though, MS has Visual Studio Express which would let you develop a desktop app to do this kind of thing, but that just feels like overkill.

Maybe something like Filemaker Pro?

[calumniate]

Question - how 'real time' do you want this? Do you want to have analysts working on it or do you want the software to that part? As in, are you forecasting for next week or are you trying to do 1 minute trading?

[Slava]

Well the real time isn't very important at all. The criteria I want to filter and rank by is all on the financial statements and they are only updated a few times a year. I can updated prices as need be, so that's not a big deal.