Reading all text files in directory
Reading all text files in directory
Reading all text files in directory
Reading all text files in directory
Reading all text files in directory
Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory
Reading all text files in directory Reading all text files in directory
Reading all text files in directory
Go Back  Xtreme Visual Basic Talk > > > Reading all text files in directory


Reply
 
Thread Tools Display Modes
  #1  
Old 02-25-2013, 04:15 PM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default Reading all text files in directory


Presently this code opens one .txt file at time, reads the data and then populates a user interface:
Code:
        OpenFD.InitialDirectory = "C:\Program Files (x86)\EPC\Ace"
        OpenFD.Title = "Open a Text File"
        OpenFD.Filter = "Text Files|*.txt|All Files|*.*"
        OpenFD.FileName = ""
        If OpenFD.ShowDialog() = Windows.Forms.DialogResult.OK Then
            Dim Lines = _
        ( _
            From line In IO.File.ReadAllLines(OpenFD.FileName) _
            Where line.Length > 0 AndAlso _
            Not line.StartsWith("5") AndAlso _
            Not line.StartsWith("9") _
        ).ToList
            For Each line In Lines
                txtAccountNumber.Text = line.Substring(21, 4)
                txtAmountPaid.Text = line.Substring(62, 7)
            Next
       End If
Then the data is saved to an Access DB.

Since many txt files are in the directory, it is desirable to read them without using an OpenFileDialog. Eliminate the .showdialog and add a
For each
'reading each text file, extracting the data as above and then saving the data before moving to the next text file. the if then end if would be removed.

However, my level of expertise is limited on how to grab each text file. Any help would be great. Thanks.
Reply With Quote
  #2  
Old 02-25-2013, 06:23 PM
AtmaWeapon's Avatar
AtmaWeaponReading all text files in directory AtmaWeapon is offline
Fabulous Florist

Forum Leader
* Guru *
 
Join Date: Feb 2004
Location: Austin, TX
Posts: 9,500
Default

The Directory.GetFiles() method returns an array of filenames that match a pattern.
__________________
.NET Resources
My FAQ threads | Tutor's Corner | Code Library
I would bet money 2/3 of .NET questions are already answered in one of these three places.
Reply With Quote
  #3  
Old 03-01-2013, 06:23 AM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

thank you. I will give it a try.
Reply With Quote
  #4  
Old 07-12-2013, 06:14 AM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

Here is what I am trying to use to read the files in a directory all at one time:


Code:
        Dim sMsg As String = "Add all records?"
        If MsgBox(sMsg, MsgBoxStyle.YesNo, "Add All Records") = MsgBoxResult.Yes Then

            Dim di As New IO.DirectoryInfo("C:\Program Files (x86)\Everybody's Phone Company\Ace")
            Dim fiGetFiles As IO.FileInfo() = di.GetFiles
            Dim fi As IO.FileInfo

            For Each fi In fiGetFiles

                Dim Lines = _
       ( _
           From line In IO.File.ReadAllLines(OpenFileDialog1.FileName) _
           Where line.Length > 0 AndAlso _
           Not line.StartsWith("5") AndAlso _
           Not line.StartsWith("9") _
       ).ToList


                For Each line In Lines
                    txtAccountNumber.Text = line.Substring(21, 4)
                    txtAmountPaid.Text = line.Substring(62, 7)
                    txtAmountPaid.Text = Replace(LTrim(Replace(txtAmountPaid.Text, "0", " ")), " ", "0")
                    Dim sString As String = txtAmountPaid.Text

                    Dim sRight As String = Microsoft.VisualBasic.Right(sString, Len(sString) - 2)
                    Dim sLeft As String = Microsoft.VisualBasic.Left(sString, Len(sString) - 2)
                    txtAmountPaid.Text = FormatCurrency(sLeft & "." & sRight, 2)
                    txtLastName.Text = line.Substring(72, 15)
                    txtFirstName.Text = line.Substring(87, 15)
                    txtStreetNumber.Text = line.Substring(102, 5)
                    txtStreetNumber.Text = Replace(LTrim(Replace(txtStreetNumber.Text, "0", " ")), " ", "0")
                    txtAddress.Text = line.Substring(107, 20)
                    txtCity.Text = line.Substring(132, 15)
                    txtState.Text = line.Substring(147, 2)
                    txtZip.Text = line.Substring(149, 5)
                    mskPhone.Text = line.Substring(154, 10)
                    FileSave()

                Next
            Next
        Else

        End If
The part in the for each fi in fiGetFiles is also listed above which reads one text file at a time. Apparently don't need the openfiledialog in this part nor is it wanted. But, I am not sure on how to proceed.

Any help would be welcomed. thank you.
Reply With Quote
  #5  
Old 07-12-2013, 07:23 AM
AtmaWeapon's Avatar
AtmaWeaponReading all text files in directory AtmaWeapon is offline
Fabulous Florist

Forum Leader
* Guru *
 
Join Date: Feb 2004
Location: Austin, TX
Posts: 9,500
Default

I'm not sure how to proceed either. You didn't ask a question.
__________________
.NET Resources
My FAQ threads | Tutor's Corner | Code Library
I would bet money 2/3 of .NET questions are already answered in one of these three places.
Reply With Quote
  #6  
Old 07-12-2013, 11:28 AM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

the question is what goes in place of openfiledialog1.filename?
Code:
For Each fi In fiGetFiles

                Dim Lines = _
       ( _
           From line In IO.File.ReadAllLines(OpenFileDialog1.FileName) _
           Where line.Length > 0 AndAlso _
           Not line.StartsWith("5") AndAlso _
           Not line.StartsWith("9") _
       ).ToList


                For Each line In Lines
                    'read file and save to db
                Next
            Next
After Dim lines = and the succeeding line of
From line In IO.File.ReadAllLines(OpenFileDialog1.FileName), there needs to be a filename. What should replace this --> IO.File.ReadAllLines(OpenFileDialog1.FileName)?

thank you.
Reply With Quote
  #7  
Old 07-12-2013, 11:57 AM
passel's Avatar
passelReading all text files in directory passel is offline
Sinecure Expert

Super Moderator
* Guru *
 
Join Date: Jun 2003
Location: Upstate New York, usa
Posts: 8,024
Default

I would assume you want to use fi.Name or fi.FullName
__________________
There Is An Island Of Opportunity In The Middle of Every Difficulty.
Miss That, Though, And You're Pretty Much Doomed.
Reply With Quote
  #8  
Old 07-12-2013, 12:26 PM
Gruff's Avatar
GruffReading all text files in directory Gruff is offline
Bald Mountain Survivor

Retired Moderator
* Expert *
 
Join Date: Aug 2003
Location: Oregon, USA - deceased
Posts: 6,440
Default

I believe this is what you are after. Although I cannot speak for your Linq. I haven't tried it with lines of text from a file.

Personally I wouldn't be pushing each file's content into controls and then to the database. I would write a class that when filled could be pushed into the database and use that class for both situations.

Code:
Public Class Form1 Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click Dim sMsg As String = "Add all records?" If MsgBox(sMsg, MsgBoxStyle.YesNo, "Add All Records") = MsgBoxResult.Yes Then 'Define your Folder Dim sFolder As String = "C:\Program Files (x86)\Everybody's Phone Company\Ace" 'Define an array of filenames from your folder assuming they end in .txt Dim sFiles() = IO.Directory.GetFiles(sFolder, "*.txt") For Each sFile As String In sFiles Dim Lines = ( _ From line In IO.File.ReadAllLines(sFile) _ Where line.Length > 0 AndAlso _ Not line.StartsWith("5") AndAlso _ Not line.StartsWith("9")).ToList For Each line as string In Lines txtAccountNumber.Text = line.Substring(21, 4) '. '. '. FileSave() Next Next End If End Sub End Class
__________________
Burn the land and boil the sea
You can't take the sky from me


~T
Reply With Quote
  #9  
Old 07-12-2013, 03:35 PM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

Gruff,

Many thanks for your help. Works as intended.

Now concerning the class which I have as cCustomers. Are you suggesting that I do:
Code:
dim nc as new cCustomers 
    txtAccountNumber.Text = line.Substring(21, 4)
    nc.AccountNumber = txtAccountNumber.text
    'et al
This adds another line for each account. Why is this, if correct, better?

Thank you.
Reply With Quote
  #10  
Old 07-12-2013, 04:08 PM
Gruff's Avatar
GruffReading all text files in directory Gruff is offline
Bald Mountain Survivor

Retired Moderator
* Expert *
 
Join Date: Aug 2003
Location: Oregon, USA - deceased
Posts: 6,440
Default

As your paradigm has changed from one text file at a time to many at one time does filling the textbox make sense at all?

Doesn't FileSave save to a database?

Currently you have nothing pausing each time a file finishes being read.

All the files will be pushed into the database at once. Only that last to be read will be displayed in your textboxes.

---

If it were me I'd just bypass the textboxes and dump all the data directly to the database.

Use a couple of buttons to read from the database to your controls use a few more to move forward and back through your database records. Edit and save them when needed.

In this scenario you would not need a class.
__________________
Burn the land and boil the sea
You can't take the sky from me


~T

Last edited by Gruff; 07-12-2013 at 04:13 PM.
Reply With Quote
  #11  
Old 07-14-2013, 12:30 PM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

Gruff,

Many thanx for your suggestions. The program now 'dumps' the data directly to the database. That solution, however, presented another concern. Along with the devious route to the db, filling the text boxes allowed a visual of the reading process. With that no longer visible, I have added a toolstriplable with a message.

Also added a counter and then a message box to indicate the number of records added. Unfortunately, the placement of the message box is a concern. That is as the text files are being read and then the db is ready to be updated or saved to, I am using a If then else end if clause such as:
Code:
            Me.StatusStrip1.Visible = True
            me.ToolStrip1.Visible  = True
            Dim I as integer = 0
            Dim sFiles() = IO.Directory.GetFiles(sFolder, "*.txt")
            For Each sFile As String In sFiles
                Dim Lines = ( _
                  From line In IO.File.ReadAllLines(sFile) _
                  Where line.Length > 0 AndAlso _
                  Not line.StartsWith("5") AndAlso _
                  Not line.StartsWith("9")).ToList
                For Each line In Lines
                    i = i + 1
                    With nc
                        .AccountNumber = line.Substring(21, 4)
                        'et al
                        
                        If dataMgr.FindDupe(Trim(.Lastname), Trim(.Firstname), iID) Then
                            dataMgr.UpdateCustomers(iID, nc)
                        Else
                            dataMgr.Add(.AccountNumber, .AmountPaid, .Lastname, .Firstname, _
                                        .StreetNumber, .Address, .City, .State, _
                                        .Zip, .Phone)
                        End If
                    End With
                Next
            Next
            MsgBox(i & " records added", MsgBoxStyle.OkOnly, "")
Seems that it has to be after all records are read, but the db is updated after each record. Possible to place the message box so that it shows only when adding new records?

Just checked the tool strip label and it appears as a sometime thing meaning that the status strip appears only the first time the procedure is invoked. And yet, when using the tool strip for another purpose, it works correctly. How to make it work for the right way in both places?

Thank you.

Last edited by kimt; 07-14-2013 at 02:53 PM.
Reply With Quote
  #12  
Old 07-15-2013, 12:44 PM
Gruff's Avatar
GruffReading all text files in directory Gruff is offline
Bald Mountain Survivor

Retired Moderator
* Expert *
 
Join Date: Aug 2003
Location: Oregon, USA - deceased
Posts: 6,440
Default

Program design is up to you. You have many options.

If your data is not clean and you need to edit it before you push it into the database then another option might be to read your list of filenames into a listbox.
If you click on the listbox filename your program could load that file into the textboxes for you to make changes if you need to.

Other buttons would add or update the db.

It's totally up to you.
__________________
Burn the land and boil the sea
You can't take the sky from me


~T
Reply With Quote
  #13  
Old 07-15-2013, 03:32 PM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

Thank you for your reply.

Program has a list view which shows all records. Click on any record and it fills the text boxes for editing. However, at this moment that is not my concern.

I want to alert the user to be patient while another form is loading/action is taken. I.e. while cycling through the text files to import, the time could be lengthy. So, a status strip was added in which is a tool strip. When the process begins the tool strip is supposed to appear and say something like 'be patient, grasshopper'. But, it is not working as intended.
Code:
    Private Sub mnuFileOpen_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles mnuFileOpen.Click, tsOpen.Click
        Dim sMsg As String = "Add all records?"
        Dim i As Integer = 0
        If MsgBox(sMsg, MsgBoxStyle.YesNo, "Add All Records") = MsgBoxResult.Yes Then
            Me.StatusStrip1.Visible = True   '
            Me.toolstripWait.Visible = True   ' make tool strip visible
            Me.toolstripWait.Text = "... adding records."    'give a message
            'Define your Folder
            Dim sFolder As String = "C:\Program Files (x86)\Everybody's Phone Company\Ace"
           ' complete the import
               Next
            Next
            Me.StatusStrip1.Visible = False   'close status strip
            ' Me.waitTS.Visible = False   'close tool strip
            Me.toolstripWait.Text = ""
            MsgBox(i & " records added", MsgBoxStyle.OkOnly, "Import")
Can't seem to figure out why it does not work here but it does here:
Code:
Private Sub mnuReport_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles mnuReport.Click
            Me.StatusStrip1.Visible = True
            Me.toolstripWait.Visible = True
            Me.toolstripWait.Text = "...loading report"
            With frmReport
                .Refresh()
                .StartPosition = FormStartPosition.CenterScreen
                .Width = 875
                .Height = 1100
                .Text = "New Customers"
                .Top = 25
                .Show()
            End With
        Me.toolstripWait.Visible = False
        Me.StatusStrip1.Visible = False
Any ideas would be helpful. Thank you.
Reply With Quote
  #14  
Old 07-15-2013, 04:09 PM
Gruff's Avatar
GruffReading all text files in directory Gruff is offline
Bald Mountain Survivor

Retired Moderator
* Expert *
 
Join Date: Aug 2003
Location: Oregon, USA - deceased
Posts: 6,440
Default

Really? I cannot imagine the process taking that long. How many text files are we talking ballpark per import? 10, 100?, ... We should be talking a second or two.

If it is taking a long time you might not be using the most efficient method of writing to the DB.

I cannot comment on your code as I do not know where it fits into your scheme.

I can tell you that when you communicate with a database typically your infterface will freeze until the database process is complete. As I understand it it is because you have one thread that is doing everything serially.
__________________
Burn the land and boil the sea
You can't take the sky from me


~T
Reply With Quote
  #15  
Old 07-16-2013, 07:52 AM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

Thanks for all your help. I have resolved the issues.
Reply With Quote
  #16  
Old 07-16-2013, 08:06 AM
DrPunk's Avatar
DrPunkReading all text files in directory DrPunk is offline
Senior Contributor

* Expert *
 
Join Date: Apr 2003
Location: Never where I want to be
Posts: 1,403
Default

Quote:
Originally Posted by kimt View Post
Thank you for your reply.

Program has a list view which shows all records. Click on any record and it fills the text boxes for editing. However, at this moment that is not my concern.

I want to alert the user to be patient while another form is loading/action is taken. I.e. while cycling through the text files to import, the time could be lengthy. So, a status strip was added in which is a tool strip. When the process begins the tool strip is supposed to appear and say something like 'be patient, grasshopper'. But, it is not working as intended.
Code:
    Private Sub mnuFileOpen_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles mnuFileOpen.Click, tsOpen.Click
        Dim sMsg As String = "Add all records?"
        Dim i As Integer = 0
        If MsgBox(sMsg, MsgBoxStyle.YesNo, "Add All Records") = MsgBoxResult.Yes Then
            Me.StatusStrip1.Visible = True   '
            Me.toolstripWait.Visible = True   ' make tool strip visible
            Me.toolstripWait.Text = "... adding records."    'give a message
            'Define your Folder
            Dim sFolder As String = "C:\Program Files (x86)\Everybody's Phone Company\Ace"
           ' complete the import
               Next
            Next
            Me.StatusStrip1.Visible = False   'close status strip
            ' Me.waitTS.Visible = False   'close tool strip
            Me.toolstripWait.Text = ""
            MsgBox(i & " records added", MsgBoxStyle.OkOnly, "Import")
Can't seem to figure out why it does not work here but it does here:
Code:
Private Sub mnuReport_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles mnuReport.Click
            Me.StatusStrip1.Visible = True
            Me.toolstripWait.Visible = True
            Me.toolstripWait.Text = "...loading report"
            With frmReport
                .Refresh()
                .StartPosition = FormStartPosition.CenterScreen
                .Width = 875
                .Height = 1100
                .Text = "New Customers"
                .Top = 25
                .Show()
            End With
        Me.toolstripWait.Visible = False
        Me.StatusStrip1.Visible = False
Any ideas would be helpful. Thank you.
I have highlighted the line of text (in red) that makes the second work.

Unless told otherwise it'll wait until it has finished your code before bothering to refresh the form (your code ties up the thread that the form is running on). So in the first example you hide the label before it gets a chance to refresh it.

Ultimately you want to do your text file lookup on a seperate thread to get it working proper, like. But that comes with a whole new set of complications.
__________________
There are no computers in heaven!
Reply With Quote
  #17  
Old 07-16-2013, 09:22 AM
kimt kimt is offline
Centurion
 
Join Date: Mar 2003
Posts: 144
Default

Thanks for the reply.

Placed the toolstrip and statusstrip in the first example at the beginning of the code in mnuFileOpen. Not the neatest solution but it works. The toolstrip is hidden after all records have been imported.

The refresh statement is to update the Crystal Report data.
Reply With Quote
  #18  
Old 07-16-2013, 09:43 AM
DrPunk's Avatar
DrPunkReading all text files in directory DrPunk is offline
Senior Contributor

* Expert *
 
Join Date: Apr 2003
Location: Never where I want to be
Posts: 1,403
Default

Edit :- Removed my post cos it's not even Me.Refresh that's being called.

Very sorry. Poor reading comprehension today. Please ignore anything I've posted.
__________________
There are no computers in heaven!
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Advertisement:





Free Publications
The ASP.NET 2.0 Anthology
101 Essential Tips, Tricks & Hacks - Free 156 Page Preview. Learn the most practical features and best approaches for ASP.NET.
subscribe
Programmers Heaven C# School Book -Free 338 Page eBook
The Programmers Heaven C# School book covers the .NET framework and the C# language.
subscribe
Build Your Own ASP.NET 3.5 Web Site Using C# & VB, 3rd Edition - Free 219 Page Preview!
This comprehensive step-by-step guide will help get your database-driven ASP.NET web site up and running in no time..
subscribe
Reading all text files in directory
Reading all text files in directory
Reading all text files in directory Reading all text files in directory
Reading all text files in directory
Reading all text files in directory
Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory Reading all text files in directory
Reading all text files in directory
Reading all text files in directory
 
Reading all text files in directory
Reading all text files in directory
 
-->