log file maintenance

I am writing to a log file in my ap. When the file gets too big, I would like to delete the oldest entries. Is there a quick way to do this or do i have to read the whole file, delete the first n lines and then write it back ?

I'm pretty sure if there is a way, its not really easy or quick. Reading the file into your program then deleting lines from the beginning is probably your best bet.

Thanks jpaugh
One more question - is it ok to read the file into a string array and then write back part of the array ? I am thinking of limiting the file to 1000 lines.

Here is a sample of how you may be able to pull off what you are trying to do.

Option Explicit
Dim col As Collection
Dim strLog As String

Private Sub Command1_Click()
' In this example I'm just using a command button and a text box.
' This could also be placed in an error trap instead or called
' anywhere you want to log an event.

If Text1.Text <> "" Then
WriteEvent Text1.Text
End If
End Sub

Private Sub Form_Load()
'Path to your log file
strLog = "c:\testlog.log"
End Sub

Private Function LoadLog(ByVal LogPath As String)
Dim ff As Integer
Dim strLine As String

'Get an open file number
ff = FreeFile

'Check that the collection does not exits
If Not col Is Nothing Then Set col = Nothing

'Create a collection
Set col = New Collection

'Open the log file
Open LogPath For Input As #ff
'Get each line of the log file
Do While Not EOF(ff)
Line Input #ff, strLine
'Add each line to the collection
col.Add strLine
Loop
Close #ff

End Function

Private Sub SaveLog(ByVal LogPath As String, ByVal Lines As Integer)
Dim ff As Integer
Dim ub As Integer
Dim i As Integer

'Get an open file number
ff = FreeFile

'Find the number of lines to save
ub = IIf(col.Count > Lines, Lines, col.Count)


'Open the log file
Open LogPath For Output As #ff
'Save the items in the collection
For i = 1 To ub
Print #ff, col(i)
Next i
Close #ff

'Destroy the collection
Set col = Nothing
End Sub

Private Sub WriteEvent(ByVal strEvent As String)
'Here is where you are actual writing to the log.

'load the lines in the log into a collection
LoadLog strLog
'Add the new entry to the start of the collection
col.Add strEvent, , 1
'Save the new log. The second param is the number of lines to save
SaveLog strLog, 10
End Sub

Thanks Mark, much appreciated :p

One minor correction:

'to add to the collection
col.Add strLine, cStr(counter)
'to read from the collection
Print #ff, col(cStr(i))

Collection elements should if at all possible be used with a KEY, even if often the key is just cStr(index). There are two reasons for this:

Retrieving the elements by KEY is significantly faster than retrieving them by index. For 10,000 elements, the difference is already a factor of 10 and it gets worse the larger the collection gets. Read more here: 227826
Collections are insertable, meaning that e.g. if you add "Ant", "Beetle", it is easily possible to insert "Chicken" in the middle, so that if you try to retrieve the second element, this will now he "Chicken" and no longer "Beetle".
Even though you are only going to append elements in your project, it is good practice to always use keys to read elements from the collection in the order you added them. :)

I agree that it is usually a good idea to specify a key. But in this case since you are not really searching the collection or trying to only insert unique values then I skipped the optional "Key" parameter. All I was shooting for here is to insert then new value at the begining of the collection. By passing 1 as the "Before" parameter of the "Add" method you accomplish that. Then it is a simple matter of looping through the collection using the Index to write it back to the file.

Since sefton was only talking about 1000 entries, it didn't seem like this would cause many problems preformance wise.

Yes, your example code works fine, no question about that.
But "good practice" is an important consideration, in explaining to others even more so than in you own code.
'nuff said :)

A method I use sometimes is use a "fixed width" string as my logfile entry. This way I can just truncate the file based on a CONST Length.

I also like using this VS. Append as I can insert my new log entries at the TOP of the file instead of at the bottom.

Anyway, here's a way to do it using the fixed string width and keeping the log file to fixed size (or smaller).

Const LINEBUFFSIZE As Long = 80 'Make it however long you need your line
'MAXLOGSIZE will ((LINEBUFFSIZE + 2) * max lines you want)
'10 lines for this example
Const MAXLOGSIZE As Long = 820
'Of course your log will actually have 1 line more than using this math..
'If you want your END RESULT to be MAXLOGSIZE, you just subtract from
'MAXLOGSIZE constant or something
Const LOGFILE = "C:\Text.Log"

Private Sub Command1_Click()
'Static variable is just for test purposes
Static logCounter As Long

'Use fixed width string for your new line
Dim NewLogEntry As String * LINEBUFFSIZE
Dim LogContents As String

'Just some test code.
'Here, do whatever to fill your newLogEntry buffer
logCounter = logCounter + 1
NewLogEntry = "Log File: Let's fill the buffer with 80 characters per line so - this is Line " & logCounter

'Make sure the LOGFILE exists.
If Dir$(LOGFILE) <> "" Then
'Dump the current file into a string
Open LOGFILE For Input As #1
LogContents = Input$(LOF(1), 1)
Close #1
End If

'If the File is larger than your max lines, truncate it
If Len(LogContents) > MAXLOGSIZE Then
'Add + 1 to the end to remove the leading LineFeed
LogContents = Mid$(LogContents, Len(LogContents) - MAXLOGSIZE + 1)
'Or use Right or Left.. whatever suits you
'LogContents = Right$(LogContents, MAXLOGSIZE)
End If

'Open the file back up and put your new entry in it
Open LOGFILE For Output As #1
'Using this method, you can actually "flip" how data goes in.
'Meaning you can print the "newest" log entry FIRST vs. Last like Append always does.
Print #1, LogContents & NewLogEntry
Close #1
End Sub

Now to reverse how the data goes in, you just have to change the constant numbers a little... and manually insert a linefeed between the two.

Plus, this will save you from doing a bunch of looping. As it looks above, you have to loop through it twice.

Cheers.

This seems like a good method :)
But, why do you have to use fixed-length strings? Unless I'm overlooking something, you could just do the chopping like this

newStart = InStr$(Len(LogContents) - MAXLOGSIZE + 1, LogContents, vbCrLf)
LogContents = Mid$(LogContents, newStart + 1)

to remove this limitation... (?)

Most times when I do a log file, each line starts with a date/time of the entry. Using some of the code provided by others here, you could start reading the file line-by-line and re-write each line to a different file until a certain date has been reached. Usually I need something like this when I have to keep logs from one "period" to another seperate.
Since most of my stuff runs in multi-user networks, I like to start each line with:

Date, Time, Username, then whatever log message text applies... & vbcrlf

It just helps when you're tracing a problem indicated by a log entry to when and who as well as what happened.

This seems like a good method :)
But, why do you have to use fixed-length strings? Unless I'm overlooking something, you could just do the chopping like this

newStart = InStr$(Len(LogContents) - MAXLOGSIZE + 1, LogContents, vbCrLf)
LogContents = Mid$(LogContents, newStart + 1)

to remove this limitation... (?)
That makes good sense too. I will adapt that.

I originally came up with the fixed width strings because my entries were pretty static. Like roofrabbit said, date + time + other static data + quick message, like "xx records processed successfully" or something.

I guess it all boils down to trying to manage log file "size" by bytes, not "lines". Just finding a place in your logfile and hunt the end of that line will do just dandy. :)

By roofrabbit's notes, I suppose you could just determine how far back a "date" you wanted to go back to... and just hunt that date down with InStr() and chop everything past that.