macOS Where do I start (want to build stock-market app)?

[hypomojo]

Am hoping to learn as I go as i've read tons but am running up against a gap between knowing and doing.

To start, I want to create and application that will go grab historical information for a set of stocks and then daily grab the closing information as well. I then want to calculate certain technical indicators such as certain moving averages and buy/sell indicators that use these averages. Then I'd like the application to "pick" a stock based on the data set.

I have a bunch of other stuff I want to do but first I want to get some or all of this happening. I believe this small set should help cement my head knowledge and get me moving in the right direction. I'm hoping it is finite enough to be relatively easy for a network geek turned management who now has some time on his hands after about 15 years and wants to get back to some technical work.

From what I read the xCode platform provides the ability to do this within the Mac framework. I have an ADC membership through my company and am fairly comfy in the UNIX environment but past shell scripting and some limited Perl/MySQL stuff I'm trying to forge a new path. make sense?

I've read much of the ADC tutorial and architecture documentation. I read, that's what I do but I want to start... Can you guru's shed some light?

Thanks in advance.

 

[pilotError]

It sounds like your going to build a spider or web crawler to grab stock info off of certain web sites.

Not difficult to do, I've built them in Java. You could use perl or any scripting language that has an API to access to the web.

One word of caution however, read the web sites terms that you plan on getting the data from. Most sites don't like web crawlers pulling their data (especially financial data). They will explicitly state it somewhere or another. I thought there might be a certain html tag (I'm not a web developer) that web crawlers should respect as well. Hopefully some web folks will jump in here and offer some advice from that perspective.

Once you have that working, its just a matter of getting it into a data store (database or custom file organization) and then writing your stock picking algorithms.

 

[mduser63]

What language do you want to work in?

Xcode lets you develop in a bunch of different languages, but in a way it's sort of tailored for Objective-C/Cocoa development. If you want to learn to develop Mac software that's probably the way to go anyway.

 

[exabytes18]

Awhile back, I made a Java program that simply queried yahoo.com everyday the market was open and dumped the stock prices into a MySQL database.

http://quote.yahoo.com/d/quotes.csv?s=AAPL&f=l1&e=.csv //fetches price only
http://quote.yahoo.com/d/quotes.csv?s=AAPL&f=sl1d1t1c1ohgv&e=.csv //A bunch more information

's' is the stock symbol and 'f' defines the format of the .csv file.

 

[hypomojo]

Nothing like that. I've got a source that I can, for free, download a daily CSV or XML file for all stocks for a given market (NASDAQ for example) for just such a purpose. They also provide subscription based data from point in time to real time data but for my purposes the free daily wrap up is plenty. I also have an Excel macro that will collect similar data from Yahoo finance.

As far as a language goes, I don't know. The daily file seems easy - Applescript to wget or ftp the file. I've worked with Perl and MySQL in the past to suck sysllog data off firewalls, IDS and load balancers and correlate information so that's sort of what is in the back of my head as far as a similar capability. I came up with the concepts and new what data I was looking for while my buddy the Perl guru wrote the scripts and we figured out the MySQL together... Again, pretty much a newbie on the Mac but fairly comfy on Linux/BSD/Solaris and Windows.

Next, what's the best way to store that data in a "native" Mac manner? MySQL? Is data storage/manipulation native to Carbon/Cocoa or what else?

Then, what's the best way to update that data and run the formulas to calculate say a 20, 50 and 200 day moving average for each stock symbol? Shell scripting to run a file diff and Perl to lop of what isn't needed is what I know would do it but it isn't very elegant.

Then I want an interface other than the CLI. I'm sure I can hack it together with Perl, MySQL and some shell scripting but that's not what I want to do. Aqua, Carbon, Cocoa and all the mystique and coolness... that's what I'm trying to figure out and would prefer to learn/use if possible. Perhaps I should be attempting to do something different - when you base a direction on an incorrect assumption you're pretty much screwed sort of thing.

I'm not sure what the equivalent tech/tools are - I am familiar with .NET, SQL, Perl and UNIX shell scripting but the Apple lingo isn't the same. I'm hoping I don't need to go sign up for CS101 and start with Assembly and BASIC again.

 

[Eraserhead]

Next, what's the best way to store that data in a "native" Mac manner? MySQL? Is data storage/manipulation native to Carbon/Cocoa or what else?

As long as you only need Tiger support CoreData will do it, it stores as SQL, XML or Binary (i'd recommend XML at least while your changing stuff in your application) with no extra work.

EDIT: To explain myself better, for Mac only stuff you probably want to use Cocoa, Core Data is a subset which you should learn after learning Cocoa, with a book like Cocoa: Programming for OS X by Aaron Hillegass, and there is an easy 30 minute tutorial of core data, once you've mastered the basics on Apple.com,

 

[AlmostThere]

Next, what's the best way to store that data in a "native" Mac manner? MySQL? Is data storage/manipulation native to Carbon/Cocoa or what else?

The best way should be determined by your data and from what you have suggested, it should definitely go into a database. You will probably want to select groups and sections of data, and run you different predictions / analyses at different times and compare results between times / date ranges / stocks etc. One of the most important features will likely be making your data manageable and accessible - do not underestimate this and use a tool designed for the job. (This is statement is bias by my own experience - it just sound like you are working similar sort of stuff as I do).

If I am way off base here, I have obviously mis-understood your descriptions, so skip to someone else's post ;-)

Then, what's the best way to update that data and run the formulas to calculate say a 20, 50 and 200 day moving average for each stock symbol? Shell scripting to run a file diff and Perl to lop of what isn't needed is what I know would do it but it isn't very elegant.

Download the data everyday through a cron job and load it into the database. wget, as you mentioned, and SQL will meet your needs and you can just trim data you don't need inside the database - firstly loading everything into a "load" table and then transferring relevant data into a "live" table.

In addition, this does mean that your app is going to need a good database connectivity layer. If you choose one database management system, this is less of a problem (MySQL will probably be fine). If you want to connect to multiple data sources, IMHO, you are really looking at Java. Nothing else has the flexibility to connect to different DB platforms on the Mac.

What are the formulae? You can do a lot of calculations through SQL and putting these in stored procedures may well be the best option. You may want to check out the database functionality in different platforms. Call the procedure and read the results from a table - you could even do the same thing using views.

All computing languages will do basic mathematics but if you want anything more sophisticated (in terms of calculations), you want something with easy access to decent maths / scientific libraries. If you don't want to go the SQL route, many of these libraries will automatically do basic calculations like moving averages, regression and histograms for you.

If you are anything like me, you are going to need to need some sort of visualisation system - drawing graphs and such like. Easily rendering a series of data visually will be a high priority and again you want to get this through a library. A picture paints a thousands words, as they say, and as a rule of thumb, you want something that will do this in only a handful of lines of code.

With respect to these requirements, I would suggest trying

Java - using JDBC and any database, a suitable package from http://math.nist.gov/javanumerics/ and, I suggest, JFreeChart

Python - using MySQL or Postgre, NumpPy / SciPy and matplot (look at http://pythonmac.org/packages/)

at least to PROTOTYPE your application's functionality. I don't know if Cocoa offers suitable libraries for what you want to do - there is third party MySQL library for it, and while it offers some good drawing functions, I don't know how far these go down the route of producing decent graphs for minimal effort (comparable the JFreeChart, for example - in fact I would be interested to know just how much is available if anyone cares to fill in the blanks). There is no end to the maths libraries accessible through a C interface but stuff like the CLAPACK interface offered in Apple's vecLib, while very fast to run, is slow to develop with unless you have substantial experience.

 

[Aniej]

I have kind of a bold and random question, how do you make a website that would crawl other websites? Essentially I am asking how to make google, but clearly it does not have to be as sophisticated code as the google boys put together, but what language is their engine made in and is it even plausible that I could begin making a "cr*ppy" version to search for a set of information? Also from what it sounds like, there is a clear difference between making an application to run on m mac and writing code so as to make a search engine. Does Xcode only work for making an application? Or would it work be applicable to making what I am asking about? Also, what if I wanted to make an application to do something like this, then Xcode? Sorry for what I am sure are really stupid questions, but I am trying to learn and just wanted to know what were the right steps to take?