NEWBIE: A program that runs & extracts certain data from log file

Hi All,

I am new to VB and wanting to built an application that finds errors in my log files and then it tells me which shops had those errors. To give you better idea please see my log file attached (attached as image).

i need to find certain error messages from that log file & extract the information RELATED TO THOSE ERRORS (i don't need to extract those errors themselves)



Where - how do i start? Is there a program or code that is already there that i can use?
Pointing me to the direction will be much appreciated.
Feel free to ask if you need more info or even the log file.
Thanks
-S

A big question.....What actually generates the log files?

If it is some other process maybe it would be easier to just clone the file pulldown process into your own VB program. Then you can download the files and manage the success / failure directly instead of through these log files.

A big question.....What actually generates the log files?

If it is some other process maybe it would be easier to just clone the file pulldown process into your own VB program. Then you can download the files and manage the success / failure directly instead of through these log files.

It is actually a unix program (designed 15 years ago) which pulls the sales files & generates the log files.

So i do not think there is a way we can do as you just wrote above. (also my boss wouldn't be happy with me tempering the system that is working fine for last 15 years)

Hope it helps!
Thanks for quick reply.

A 15 year old solution is often just fine but it also may be fully ripe for review. Many things change in 15 years since 1993 and there may be superior solutions than the one that manages the sales data base.

Take a look at this carefully as there are times when the alternate solution has way lower total usage/maintenance cost than the solution held in place by the ostrich that has head buried in sand.

...but on the other hand, if it aint broke...

What you're asking for is pretty straightforward if the log files are in a consistent format. Could you post a sample or PM me and I'll show you how you can strip the bits out. How big are the log files and what do you actually want to do with the bits you strip out?

J

Thanks for reply.

The thing is my company has been using this Unix software for ages and support for the Unix system is provided by a local IT company. the software is called HALMAC. Almost everything PAYROLL - SALES REPORTS - PULLING THE DAILY SALES FILES FROM CASH REGISTERS (FROM COMPANY'S 80 RETAIL STORES) - STOCK MAINTAINANCE is based on HALMAC SERVER which was designed 15 years ago.
Upgrading to another system would mean investing at least $70,000. I really don't think the company will do it in near future. besides the system works fine. All they have to do is change the server hardware every 3 or so years (change motherboard & memory &...)

Anyway, coming back to my problem. There is only one way to find out if the sales files were pulled from the stores' cash registers - BY CHECKING THE LOG FILES. Yes, the log files are consistence.

The log files can get about 5MB in size per week. I rename the log file to backup_dd-mm-yy.log every friday, so it doesn't take ages to open, and the system creates a new log file every monday.

Yes the log files are consistence and there are not many things that change. There are only 3 different type of "text blocks" can be in the log files. They are explained in attached files.
I have attached the ACTUAL log file. I have also attached a pdf that shows what different parts of the log file mean. (see attached ZIP file).

Thank you!

Take a look at the File I/O tutorial. You'll see how to read your log line by line. There's several ways you could extract your info, and given the consistent nature of the logs it wont be too hard to pull out the shop ID's for the failing items. You could really go to town on this and extract all the data but in the first instance, it will be an interesting exercise just to do the minimum.

The tutorial's here: http://www.xtremevbtalk.com/showthread.php?t=123814

JA

Thank you Transltr.

I am checking the tutorial out.

After you see how to open & read the file...

Personally, I find that reading the entire file into a single string buffer, then using the SPLIT function to break up the string buffer into an array of string is a much more efficient (and understandable) approach. Thisapproach is discussed in the *** installment on fast I/O. 5 megabytes would be dead easy to handle with this approach.

This way, each resulting array element contains a single line from the file... Just iterate through the array, processing each line. (check for the new record marker characters. Find the Date/Shop ID, & store it, and reset the File Upload Error flag. Then, continue iterating until you've hit the next record marker. With each iteration, check to see if the string contains the string which tells you that you encountered an error. If it did, output the error information. This loop will continue until you've reached the end of the array.

A few notes. The UNIX file structure uses either a CR or a LF character only to represent a line break. Windows uses two characters - BOTH the CR and the LF character. When splitting the file, you'll need to know which was used. the INSTR function is used to locate the position of a substring within a string. If it returns a value of zero, the substring wasn't found. The MID function is used to extract a substring from a string.

Great help.
That was as helpful as the tutorial.

(I have already finished TIER-1 of Microsoft Tutorials on VB. And I am starting to read a book about VB so i know what <strings> & <variables> mean).

Thank you Loquin!