Lawrence Chan of NeoTicker answers our questions

platform related discussion

Moderator: moderators

JPT
rank: <50 posts
rank: <50 posts
Posts: 49
Joined: Mon May 22, 2006 3:26 pm
Reputation: 0
Location: Monterey, CA USA
Gender: None specified

Postby JPT » Mon Jul 24, 2006 10:49 pm

Ed:

I see you are in Orange County. Maybe sometime we can talk on the phone. I am going to start learning the C#. It is going to kill me though. Well, need to get with the 21st centuary.

Regards

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.

User avatar
eudamonia
rank: 500+ posts
rank: 500+ posts
Posts: 536
Joined: Thu Jun 15, 2006 9:50 pm
Reputation: 0
Location: Rocklin, CA
Real name: Edward Heming
Gender: Male
Contact:

Postby eudamonia » Mon Jul 24, 2006 10:58 pm

JPT,

You've got a PM.

Edward
Eudaimonia (pron.: you-die-moan-e-a) (Greek: εὐδαιμονία) is a classical Greek word commonly translated as 'happiness'. The less subjective "human flourishing" is often preferred as a translation.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Mon Jul 24, 2006 11:00 pm

eudamonia wrote:Michal,

Thanks Michal. I'm definately interested in becoming a NeoTicker convert.


Ed, when you're ready to jump the ship :), drop me a PM and I'll include you in the hidden discussion group here on the forum where we're disussing the issues of NeoTicker volume discount.

Of course, this applies only if you want to save on your potential Neo purchase :)

Michal

User avatar
eudamonia
rank: 500+ posts
rank: 500+ posts
Posts: 536
Joined: Thu Jun 15, 2006 9:50 pm
Reputation: 0
Location: Rocklin, CA
Real name: Edward Heming
Gender: Male
Contact:

Postby eudamonia » Mon Jul 24, 2006 11:41 pm

Michal,

Thanks, I'll let you know shortly.

Edward
Eudaimonia (pron.: you-die-moan-e-a) (Greek: εὐδαιμονία) is a classical Greek word commonly translated as 'happiness'. The less subjective "human flourishing" is often preferred as a translation.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Wed Jul 26, 2006 4:32 pm

Lawrence,

one simple, but important question: is it possible to store the history data to be used with NeoTicker in an SQL database?

There are free SQL servers like Microsoft SQL Express 2005.

Michal

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.

Lawrence Chan
rank: <50 posts
rank: <50 posts
Posts: 41
Joined: Sat Jun 24, 2006 1:20 pm
Reputation: 0
Location: Toronto
Gender: Male
Contact:

Postby Lawrence Chan » Wed Jul 26, 2006 4:41 pm

Real-time storage using SQL db is slow, thus that is not a good choice.

On the other hand, we've planned to open up the non real-time data source API, which will then enable plug-in data sources that NeoTicker can access data from.

For example, one of the wish items from our clients is to add support of Excel as a data source. That will be done through the plug-in architecture.
Lawrence Chan
TickQuest Inc.
http://www.tickquest.com

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Wed Jul 26, 2006 4:56 pm

Thanks for the answer, which was record-setting fast (9 minutes)!

I didn't mean to store the data in SQL in order to work with them realtime with charts, backtests etc.

Instead I was talking about how the data is stored when not in use. When there's a request for the data either to put them on chart or to perform a backtest, it must be loaded from somewhere. The best way to store structured data for later use is in my opinion the SQL DB.

The advantages of storing the history data in a single SQL DB would be:
  • making them available for third-party software or even for my own utilities all in one place
  • you can perform various data managing tasks on the SQL data like mirroring them automatically, making sure the data has integrity, backuping the entire database automatically etc.
  • I'm sure loading data from SQL is much faster than loading it from a plain file on the hard disk

Michal

Lawrence Chan
rank: <50 posts
rank: <50 posts
Posts: 41
Joined: Sat Jun 24, 2006 1:20 pm
Reputation: 0
Location: Toronto
Gender: Male
Contact:

Postby Lawrence Chan » Thu Jul 27, 2006 1:43 am

When I am in my office, Outlook reminds me of new emails all the time :)

Now, back to SQL access.

Currently, the NeoTicker cache data is the primary data source when you are working with data collected from real-time data services. Making it available is something we've be working on since v2.

We've been working with a few different ideas in making the data available,

1. an ActiveX access based on NeoTicker - not a good one because that requires opening NeoTicker. But, any data that NeoTicker can access will be available to the user ... so that is an advantage we cannot ignore.

2. a DLL that enable anyone to access NT data - that will be the fastest and low cost for development, but it will shut out the other data types from being accessible.

3. an export mechanism - which we have never touched. If the export mechanism exports into some sort of SQL db, your wish would come true already. In fact, our in house DB is based on MySQL ... and you already know that we like working with SQL db because Grid Optimizer works with SQL :)

Out of the 3 possible choices, I think users who work with multiple data sources will like #1 best. And for all others, #2 or #3 will do the job.

As usual, all suggestions are welcome.
Lawrence Chan

TickQuest Inc.

http://www.tickquest.com

aspTrader
rank: 50+ posts
rank: 50+ posts
Posts: 57
Joined: Fri May 19, 2006 5:05 am
Reputation: 0
Gender: None specified

Postby aspTrader » Thu Jul 27, 2006 2:02 am

michal.kreslik wrote:The best way to store structured data for later use is in my opinion the SQL DB.

I did Oracle RDB related for consulting for 20 years before full time focus on markets and trading so I'm a fan of relational database management and sql.

I'd be willing to bet that NeoTicker's current method for storage of tick data would be faster than any well known RDBMS including Oracle.

The partitioning of minute and tick data into separate files by day is tremendous IMO. It means that a single physical file can get corrupted without corrupting the whole data series. (Contrast this with TS where all tick and minute data is stored in single files and if the file gets corrupted you're out of luck.

But I agree that the relational model is intriguing. I have been curious about these folks for a good while... www.kx.com Relational and good performance?

One idea Lawrence might be to leverage the work of the people at kx.com and provide NT users with a choice about whether they would like to use your current data storage approach or a kx based data storage tool. Users desiring relational would pay a little more for the kx license but would get what they need.

User avatar
michal.kreslik
rank: 1000+ posts
rank: 1000+ posts
Posts: 1032
Joined: Sat May 13, 2006 2:40 am
Reputation: 0
Location: Monte Carlo, Monaco
Real name: Michal Kreslik
Gender: Male
Contact:

Postby michal.kreslik » Thu Jul 27, 2006 9:30 am

Lawrence Chan wrote:1. an ActiveX access based on NeoTicker - not a good one because that requires opening NeoTicker.

The history data should be stored in a format independent of NeoTicker running or not.

Lawrence Chan wrote:2. a DLL that enable anyone to access NT data - that will be the fastest and low cost for development, but it will shut out the other data types from being accessible.

As above, that would eliminate the interchangeability of the data.

Lawrence Chan wrote:3. an export mechanism - which we have never touched. If the export mechanism exports into some sort of SQL db, your wish would come true already.

That wouldn't bring anything new. The purpose of the data stored in an SQL would be to interchange and share it between NeoTicker and other trading-related SW. The purpose of the SQL-stored history data would be to not only "backup" the data to SQL, but to actually use the SQL DB actively by Neo and other SW as a source for history data.

Lawrence Chan wrote:In fact, our in house DB is based on MySQL ... and you already know that we like working with SQL db because Grid Optimizer works with SQL :)


I know and it's exatly why I was raising this question in the first place :) Why not go the whole nine yeards and use this same cool concept in Neo, too??

I would go for some free SQL, like as I've already expressed :) the MS SQL Express 2005. The data would be stored offline in the SQL and loaded into memory upon request. This would make it possible for the third party tools (now I'm talking about my own SW tools) to prepare and store the data into the known, standard format - SQL and make this data available for Neo and other data-mining SW, too, at the same time.

The great general idea of NeoTicker is to separate platform, data provider and a brokerage so that the user might actually choose which service provider offers the best value and functionality for him. Then, separating and in fact standardizing the format in which the history data is stored would be the logical next step for TickQuest.

Michal

Please add www.kreslik.com to your ad blocker white list.
Thank you for your support.


Return to “NeoTicker platform”