Sunday, March 25, 2012

Downloading data from our Informix server

Our entire sales database sits on an Informix server and users run their reports directly to this server.

Because of the volume of data - upwards of 7 million rows in some tables we are finding performance is becoming an issue. We realise that Informix is not the best for running queries and so are looking to replicate to a SQL server and use that as a reporting server.

My chosen method is DTS using ODBC which works fine. However due to the sheer volume of data to be downloaded each night (over a 512K wan link) we only want to download changed data.

Is there any way we can get SQL2000 to download only change data - i.e a differential download or something? We don't have a unique field in the table (for whatever reason) such as sales date or transaction ID (shops might upload old / back-dated transactions for example due to a systems failure).

If I do have to download all the data every night is my method the most efficient or is there some other approach I could use?

Ultimatly the aim here is to relieve the Informix server of the reporting overhead so any alternative ideas are welcome!

Many thanksHi
The normal way of doing this sort of thing would be replication - this would give you your "differential downlaod".
I am not sure what one can do repl wise between informix & mssql but would be worth checking it out...
Des
(otherwise can use tracking tables which record max id last copied & copy only from there each day - use vars in DTS package, seeded by values in the tracking tables)|||Which version of informix are you running - believe it or not informix is very good at running queries, so I would first check to see if either the queries are not optimized correctly, check for poor database design and also look at the onconfig file to make sure the server itself has the appropriate settings.

To solve the transfer issue - probably you best method is to dump the data and use sql server to bulk copy (bcp) or bulk insert the data. You can use some identifier within Informix such as a date/time stamp to find the changed records - however, I go back to there is probably something either designed poorly or misconfigured on your informix box that is causing this poor performance.

No comments:

Post a Comment