What method of I/O to use?
What method of I/O to use?
What method of I/O to use?
What method of I/O to use?
What method of I/O to use?
What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use?
What method of I/O to use? What method of I/O to use?
What method of I/O to use?
Go Back  Xtreme Visual Basic Talk > > > What method of I/O to use?


Reply
 
Thread Tools Display Modes
  #1  
Old 09-08-2011, 05:24 PM
MrMuffles MrMuffles is offline
Newcomer
 
Join Date: Oct 2010
Posts: 8
Default What method of I/O to use?


Hello, long-time VB6er here just now converting over to VB.net.

I'm reading up that INI isn't the way to go anymore, and seeing how there are no stickies in this subforum, I'm curious as to which method of I/O I should learn for a simple application that stores weather data.

If you need any further info, I'll be happy to clear anything up. Otherwise, thanks in advance.
Reply With Quote
  #2  
Old 09-08-2011, 06:01 PM
AtmaWeapon's Avatar
AtmaWeaponWhat method of I/O to use? AtmaWeapon is offline
Fabulous Florist

Forum Leader
* Guru *
 
Join Date: Feb 2004
Location: Austin, TX
Posts: 9,500
Default

There's no easily accessible API in .NET for parsing INI files, so yeah that's out as a general-purpose storage mechanism. But you've still got a lot of options.

If you feel like fiddling with XML there's three built-in XML parsers in .NET:
  • XmlReader/XmlWriter follow a model where you parse one element at a time from the top of the file to the bottom. This is ideal for very large data sets, but tedious.
  • XmlDocument is a DOM parser. This makes it really easy to find specific bits of information in a file. Unfortunately, it requires loading the entire file into memory at once so it can be a burden for very large data sets.
  • XDocument is a DOM parser like XmlDocument that works well with LINQ. It's got the same advantages and disadvantages; the reason to use one over the other comes down to what you want your code to look like.
XML serialization is another technique. It promises "save and load files in 2 or 3 lines" but sometimes it's not that easy. At a high level, you represent your data as objects and XML serialization knows how to convert those objects to and from XML.

Don't want to use XML? Sometimes that's a good idea. You can write your own format. If you want to work with text files, StreamWriter and StreamReader are the obvious choices. If you want to work with binary files, FileStream is probably the best choice. There is also a binary serialization framework.

There's a weird class buried in the VB-specific libraries called TextFieldParser that makes working with delimited text files like CSV easier. It doesn't work in all cases, but if you're designing your own file format you can choose a design that works well with it.

File I/O in VB .NET is a good place to start. When you can wrangle text files easily working with XML is easier.
__________________
.NET Resources
My FAQ threads | Tutor's Corner | Code Library
I would bet money 2/3 of .NET questions are already answered in one of these three places.
Reply With Quote
  #3  
Old 09-10-2011, 03:18 AM
MrMuffles MrMuffles is offline
Newcomer
 
Join Date: Oct 2010
Posts: 8
Default

Thanks for the helpful response, and sorry for the delayed reply.

What I'm going to be storing is 20+ years of weather data. It sounds big but it's not all that much, really. It'll look something like this:

Code:
August 24
1987
High: 103
Low: 77
Wind: 2 mph
Rain: 0
Obviously when stored, it'll be simplified, like,
Code:
8/24/1987
103
77
2
0
So with all this said, what would you recommend? A simple text file storage or xml?

Also, sorta off-topic, if I recall correctly, I remember having issues with the amount of data I could store in embedded arrays. This was many years ago, but I remember trying to have a class array within a class array and the program just didn't want to handle it and gave me memory error messages.

Will I run into this similar problem using VB.net? For example:
Code:
Year(1987).Month(August).Day(24).High = 103
or even
Weather(8, 24, 1987).High = 103
Right off the bat it looks like too much. If this is the case, do you have any suggestions to alleviate this problem?

Thanks again and I appreciate the helpful replies.

Last edited by MrMuffles; 09-10-2011 at 03:24 AM.
Reply With Quote
  #4  
Old 09-10-2011, 04:08 AM
PlausiblyDamp's Avatar
PlausiblyDampWhat method of I/O to use? PlausiblyDamp is offline
Ultimate Contributor

Forum Leader
* Expert *
 
Join Date: Nov 2003
Location: Newport, Wales
Posts: 2,058
Default

It really depends on what you are wanting to do with the data in the long term, if you are needing to report on this data or do a lot of querying then it might be easier to put it into a database rather than handle the storage yourself.

If you are going to be saving it into your own file format then the pros and cons of different approaches are pretty much outlined in Atma's post above.
__________________
Intellectuals solve problems; geniuses prevent them.
-- Albert Einstein

Posting Guidelines Forum Rules Use the code tags
Reply With Quote
  #5  
Old 09-10-2011, 06:48 AM
Rockoon's Avatar
Rockoon Rockoon is offline
Joseph Koss

* Guru *
 
Join Date: Aug 2003
Location: Unfashionable End
Posts: 3,615
Default

Weather data is usually already in CSV format... why not continue with CSV?
Reply With Quote
  #6  
Old 09-10-2011, 11:15 AM
MrMuffles MrMuffles is offline
Newcomer
 
Join Date: Oct 2010
Posts: 8
Default

This weather data is actually currently hand-written on paper and I have been given the lovely task of transferring it to computer. :P

CSV doesn't sound like a bad idea though. Would I just be using the basic FileStream for this?
Reply With Quote
  #7  
Old 09-10-2011, 11:49 AM
Rockoon's Avatar
Rockoon Rockoon is offline
Joseph Koss

* Guru *
 
Join Date: Aug 2003
Location: Unfashionable End
Posts: 3,615
Default

I would use FileStream or simply Stream...

CSV has the added benefit that all spreadsheets programs (that I am aware of) know how to load them and throw them into the grid.
Reply With Quote
  #8  
Old 09-10-2011, 11:57 AM
AtmaWeapon's Avatar
AtmaWeaponWhat method of I/O to use? AtmaWeapon is offline
Fabulous Florist

Forum Leader
* Guru *
 
Join Date: Feb 2004
Location: Austin, TX
Posts: 9,500
Default

FileStream and Stream are building blocks for reading files at the binary level. If you're reading a formatted text file (CSV is formatted text) it's much easier to use StreamReader since it's oriented towards reading text. The TextFieldParser class I mentioned is also ideal for reading CSV data depending on your desired format.
__________________
.NET Resources
My FAQ threads | Tutor's Corner | Code Library
I would bet money 2/3 of .NET questions are already answered in one of these three places.
Reply With Quote
  #9  
Old 09-10-2011, 12:56 PM
MrMuffles MrMuffles is offline
Newcomer
 
Join Date: Oct 2010
Posts: 8
Default

Okay, so I've decided to go with CSV and to use StreamReader (and maybe StreamWriter) as the I/O method.

The last question is basically referring back to my previous post. The memory issue. Do I load this somewhat giant CSV file into a multidimensional array? Do I load it all into memory at all? Or do I simply parse through the entire file looking for specific dates and values over and over again? There will be lots of this going on.

Appreciate the help!
Reply With Quote
  #10  
Old 09-10-2011, 01:11 PM
PlausiblyDamp's Avatar
PlausiblyDampWhat method of I/O to use? PlausiblyDamp is offline
Ultimate Contributor

Forum Leader
* Expert *
 
Join Date: Nov 2003
Location: Newport, Wales
Posts: 2,058
Default

It would probably be best to create a class to encapsulate a single weather record and when reading from the file just build a list of these classes.

That way you could query the in memory structures rather than constantly go back to the file and re-parsing the file. If the amount of data is too large to hold in memory then there is a good chance your performance will suffer if you need to keep re-reading the file, in this scenario a database may be a better option.
__________________
Intellectuals solve problems; geniuses prevent them.
-- Albert Einstein

Posting Guidelines Forum Rules Use the code tags
Reply With Quote
  #11  
Old 09-10-2011, 02:59 PM
AtmaWeapon's Avatar
AtmaWeaponWhat method of I/O to use? AtmaWeapon is offline
Fabulous Florist

Forum Leader
* Guru *
 
Join Date: Feb 2004
Location: Austin, TX
Posts: 9,500
Default

Building an efficient data structure isn't something that has general-purpose advice. PlausiblyDamp took it about as far as one can go without having a lot of details. If you represent your data as objects, it becomes easier to build data structures that answer your application's questions quickly. Picking the right design involves knowing what questions you'd like to ask.
__________________
.NET Resources
My FAQ threads | Tutor's Corner | Code Library
I would bet money 2/3 of .NET questions are already answered in one of these three places.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Advertisement:





Free Publications
The ASP.NET 2.0 Anthology
101 Essential Tips, Tricks & Hacks - Free 156 Page Preview. Learn the most practical features and best approaches for ASP.NET.
subscribe
Programmers Heaven C# School Book -Free 338 Page eBook
The Programmers Heaven C# School book covers the .NET framework and the C# language.
subscribe
Build Your Own ASP.NET 3.5 Web Site Using C# & VB, 3rd Edition - Free 219 Page Preview!
This comprehensive step-by-step guide will help get your database-driven ASP.NET web site up and running in no time..
subscribe
What method of I/O to use?
What method of I/O to use?
What method of I/O to use? What method of I/O to use?
What method of I/O to use?
What method of I/O to use?
What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use? What method of I/O to use?
What method of I/O to use?
What method of I/O to use?
 
What method of I/O to use?
What method of I/O to use?
 
-->