Ed:
I see you are in Orange County. Maybe sometime we can talk on the phone. I am going to start learning the C#. It is going to kill me though. Well, need to get with the 21st centuary.
Regards
Lawrence Chan of NeoTicker answers our questions
Moderator: moderators
Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.
Thank you for your support.
- michal.kreslik
- rank: 1000+ posts
- Posts: 1048
- Joined: Sat May 13, 2006 2:40 am
- Reputation: 36
- Location: Monte Carlo, Monaco
- Real name: Michal Kreslik
- Gender:
eudamonia wrote:Michal,
Thanks Michal. I'm definately interested in becoming a NeoTicker convert.
Ed, when you're ready to jump the ship , drop me a PM and I'll include you in the hidden discussion group here on the forum where we're disussing the issues of NeoTicker volume discount.
Of course, this applies only if you want to save on your potential Neo purchase
Michal
- michal.kreslik
- rank: 1000+ posts
- Posts: 1048
- Joined: Sat May 13, 2006 2:40 am
- Reputation: 36
- Location: Monte Carlo, Monaco
- Real name: Michal Kreslik
- Gender:
Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.
Thank you for your support.
-
- rank: <50 posts
- Posts: 41
- Joined: Sat Jun 24, 2006 1:20 pm
- Reputation: 0
- Location: Toronto
- Gender:
- Contact:
Real-time storage using SQL db is slow, thus that is not a good choice.
On the other hand, we've planned to open up the non real-time data source API, which will then enable plug-in data sources that NeoTicker can access data from.
For example, one of the wish items from our clients is to add support of Excel as a data source. That will be done through the plug-in architecture.
On the other hand, we've planned to open up the non real-time data source API, which will then enable plug-in data sources that NeoTicker can access data from.
For example, one of the wish items from our clients is to add support of Excel as a data source. That will be done through the plug-in architecture.
- michal.kreslik
- rank: 1000+ posts
- Posts: 1048
- Joined: Sat May 13, 2006 2:40 am
- Reputation: 36
- Location: Monte Carlo, Monaco
- Real name: Michal Kreslik
- Gender:
Thanks for the answer, which was record-setting fast (9 minutes)!
I didn't mean to store the data in SQL in order to work with them realtime with charts, backtests etc.
Instead I was talking about how the data is stored when not in use. When there's a request for the data either to put them on chart or to perform a backtest, it must be loaded from somewhere. The best way to store structured data for later use is in my opinion the SQL DB.
The advantages of storing the history data in a single SQL DB would be:
Michal
I didn't mean to store the data in SQL in order to work with them realtime with charts, backtests etc.
Instead I was talking about how the data is stored when not in use. When there's a request for the data either to put them on chart or to perform a backtest, it must be loaded from somewhere. The best way to store structured data for later use is in my opinion the SQL DB.
The advantages of storing the history data in a single SQL DB would be:
- making them available for third-party software or even for my own utilities all in one place
- you can perform various data managing tasks on the SQL data like mirroring them automatically, making sure the data has integrity, backuping the entire database automatically etc.
- I'm sure loading data from SQL is much faster than loading it from a plain file on the hard disk
Michal
-
- rank: <50 posts
- Posts: 41
- Joined: Sat Jun 24, 2006 1:20 pm
- Reputation: 0
- Location: Toronto
- Gender:
- Contact:
When I am in my office, Outlook reminds me of new emails all the time
Now, back to SQL access.
Currently, the NeoTicker cache data is the primary data source when you are working with data collected from real-time data services. Making it available is something we've be working on since v2.
We've been working with a few different ideas in making the data available,
1. an ActiveX access based on NeoTicker - not a good one because that requires opening NeoTicker. But, any data that NeoTicker can access will be available to the user ... so that is an advantage we cannot ignore.
2. a DLL that enable anyone to access NT data - that will be the fastest and low cost for development, but it will shut out the other data types from being accessible.
3. an export mechanism - which we have never touched. If the export mechanism exports into some sort of SQL db, your wish would come true already. In fact, our in house DB is based on MySQL ... and you already know that we like working with SQL db because Grid Optimizer works with SQL
Out of the 3 possible choices, I think users who work with multiple data sources will like #1 best. And for all others, #2 or #3 will do the job.
As usual, all suggestions are welcome.
Now, back to SQL access.
Currently, the NeoTicker cache data is the primary data source when you are working with data collected from real-time data services. Making it available is something we've be working on since v2.
We've been working with a few different ideas in making the data available,
1. an ActiveX access based on NeoTicker - not a good one because that requires opening NeoTicker. But, any data that NeoTicker can access will be available to the user ... so that is an advantage we cannot ignore.
2. a DLL that enable anyone to access NT data - that will be the fastest and low cost for development, but it will shut out the other data types from being accessible.
3. an export mechanism - which we have never touched. If the export mechanism exports into some sort of SQL db, your wish would come true already. In fact, our in house DB is based on MySQL ... and you already know that we like working with SQL db because Grid Optimizer works with SQL
Out of the 3 possible choices, I think users who work with multiple data sources will like #1 best. And for all others, #2 or #3 will do the job.
As usual, all suggestions are welcome.
michal.kreslik wrote:The best way to store structured data for later use is in my opinion the SQL DB.
I did Oracle RDB related for consulting for 20 years before full time focus on markets and trading so I'm a fan of relational database management and sql.
I'd be willing to bet that NeoTicker's current method for storage of tick data would be faster than any well known RDBMS including Oracle.
The partitioning of minute and tick data into separate files by day is tremendous IMO. It means that a single physical file can get corrupted without corrupting the whole data series. (Contrast this with TS where all tick and minute data is stored in single files and if the file gets corrupted you're out of luck.
But I agree that the relational model is intriguing. I have been curious about these folks for a good while... www.kx.com Relational and good performance?
One idea Lawrence might be to leverage the work of the people at kx.com and provide NT users with a choice about whether they would like to use your current data storage approach or a kx based data storage tool. Users desiring relational would pay a little more for the kx license but would get what they need.
- michal.kreslik
- rank: 1000+ posts
- Posts: 1048
- Joined: Sat May 13, 2006 2:40 am
- Reputation: 36
- Location: Monte Carlo, Monaco
- Real name: Michal Kreslik
- Gender:
Lawrence Chan wrote:1. an ActiveX access based on NeoTicker - not a good one because that requires opening NeoTicker.
The history data should be stored in a format independent of NeoTicker running or not.
Lawrence Chan wrote:2. a DLL that enable anyone to access NT data - that will be the fastest and low cost for development, but it will shut out the other data types from being accessible.
As above, that would eliminate the interchangeability of the data.
Lawrence Chan wrote:3. an export mechanism - which we have never touched. If the export mechanism exports into some sort of SQL db, your wish would come true already.
That wouldn't bring anything new. The purpose of the data stored in an SQL would be to interchange and share it between NeoTicker and other trading-related SW. The purpose of the SQL-stored history data would be to not only "backup" the data to SQL, but to actually use the SQL DB actively by Neo and other SW as a source for history data.
Lawrence Chan wrote:In fact, our in house DB is based on MySQL ... and you already know that we like working with SQL db because Grid Optimizer works with SQL
I know and it's exatly why I was raising this question in the first place Why not go the whole nine yeards and use this same cool concept in Neo, too??
I would go for some free SQL, like as I've already expressed the MS SQL Express 2005. The data would be stored offline in the SQL and loaded into memory upon request. This would make it possible for the third party tools (now I'm talking about my own SW tools) to prepare and store the data into the known, standard format - SQL and make this data available for Neo and other data-mining SW, too, at the same time.
The great general idea of NeoTicker is to separate platform, data provider and a brokerage so that the user might actually choose which service provider offers the best value and functionality for him. Then, separating and in fact standardizing the format in which the history data is stored would be the logical next step for TickQuest.
Michal
Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.
Thank you for your support.