Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line
Reading all txt file in a folder line by line Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Go Back  Xtreme Visual Basic Talk > > > Reading all txt file in a folder line by line


Reply
 
Thread Tools Display Modes
  #1  
Old 03-29-2012, 12:30 PM
enis enis is offline
Regular
 
Join Date: Apr 2004
Location: Turkey, Eskisehir
Posts: 52
Default Reading all txt file in a folder line by line


Hello everybody,
I am quite new in VB NET, after VB. and unfortuantely I had to copy paste and merge the code fragments from some tutorials.

I have a folder including 200 eng files which can be opened by notepad. And each file has around 4000 row; with 60 field seperated by coma. I need to read specific row-field value at each files.

Now what I can is:
a) reading individual txt files and find specific value
b) listing all of the files in the folder
c) I can merge (a) and (b) and read all the files automatically, but since ram is continuously increasing, and at one point (sometimes 163, sometimes 169, or 170. files or something) it gives an error as "System.OutOfMemoryException". My ram is 4 GB and when I get this error the ram is increasing to more than 3 GB.

Questions on my mind?
1- Could it be happen because of being eng files instead of regular txt files?
2- Why ram is continuously increasing in such a rate (from 1.6 to 3. something)?
3- What is the problem of codes?

I spent one day to solve it and failed.
Thank you in advance.

The code:

Code:
        Dim di As New IO.DirectoryInfo("c:\testfolder2\")
        Dim diar1 As IO.FileInfo() = di.GetFiles()
        Dim dra As IO.FileInfo
        Dim a(250) As String
        Dim satir As Integer = 0
        Dim strArr() As String
        Dim rowx(250, 10000, 70) As String
        Dim count As Integer

        Dim f As Integer = 0
        Dim x As Integer




        For Each dra In diar1

            a(x) = dra.FullName
            x = x + 1
            next


        For f = 0 To x - 1
            Dim sr As StreamReader = New StreamReader(a(f))
            Dim line As String
            satir = 0
            Do
                line = sr.ReadLine()
                If line = "" Then GoTo outx
                strArr = line.Split(",")


                For count = 0 To strArr.Length - 1
                    rowx(f, satir, count) = strArr(count)
                Next


                satir = satir + 1

            Loop Until line Is Nothing
outx:
            sr.Close()
        Next
Reply With Quote
  #2  
Old 03-29-2012, 02:08 PM
Gruff's Avatar
GruffReading all txt file in a folder line by line Gruff is offline
Bald Mountain Survivor

Retired Moderator
* Expert *
 
Join Date: Aug 2003
Location: Oregon, USA - deceased
Posts: 6,440
Default

enis,

So basically you are trying to load all of the text from all of the files into one huge array.
Your code does not explain what you are searching for within all this text.

What you are trying to do fairly screams "Database". I wouldn't be messing around with text files for this sort of problem.

a Database has wonderful tools built in to find anything inside it quickly,efficiently and easily.

---

Your variable definition: Dim rowx(250, 10000, 70) As String
is trying to define a cube containing 17,500,000 strings where a string can be just about any size of a single line in your text file. Look at each character in a line as a byte.
Can you see that you are using up vast amounts of memory?
__________________
Burn the land and boil the sea
You can't take the sky from me


~T
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Advertisement:





Free Publications
The ASP.NET 2.0 Anthology
101 Essential Tips, Tricks & Hacks - Free 156 Page Preview. Learn the most practical features and best approaches for ASP.NET.
subscribe
Programmers Heaven C# School Book -Free 338 Page eBook
The Programmers Heaven C# School book covers the .NET framework and the C# language.
subscribe
Build Your Own ASP.NET 3.5 Web Site Using C# & VB, 3rd Edition - Free 219 Page Preview!
This comprehensive step-by-step guide will help get your database-driven ASP.NET web site up and running in no time..
subscribe
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
 
Reading all txt file in a folder line by line
Reading all txt file in a folder line by line
 
-->