XML Creating and Parsing Very large XML Files

Hello Everyone

For a long while I have used SAX for Pascal with a Delphi 6 project.
it is reading and writing very large XML files (over 800MB and
larger).  The project has been running for years - and I recently had
to upgrade to Delphi 2007 - and I discovered that SAX for Pascal has
not been ported to D2007.  The authors have provided full source -
which I have - but now I need to get it installed into D2007.  
Has anyone moved the Packages over to D2007?  

If this is not the best way of reading and writing really large XML
files anymore - any thoughts as to what is the best method?  



The latest release of SAX for Pascal was on Ver 1.1 July 4, 2003


G. Bradley MacDonald
brad_AT_timeacct_DOT_Com
0
G
9/23/2008 6:13:25 PM
embarcadero.delphi.xml 291 articles. 0 followers. Follow

6 Replies
1756 Views

Similar Articles

[PageSpeed] 43

> {quote:title=Guest wrote:}{quote}
> Hello Everyone
> 
> For a long while I have used SAX for Pascal with a Delphi 6 project.
> it is reading and writing very large XML files (over 800MB and
> larger).  The project has been running for years - and I recently had
> to upgrade to Delphi 2007 - and I discovered that SAX for Pascal has
> not been ported to D2007.  The authors have provided full source -
> which I have - but now I need to get it installed into D2007.  
> Has anyone moved the Packages over to D2007?  
> 
> If this is not the best way of reading and writing really large XML
> files anymore - any thoughts as to what is the best method?  
> 

Brad,

I use OmniXML here. It works extremely well, is free with source, and the support for it by its' developers is fantastic. 

You can find it at http://www.omnixml.com/
0
Ken
9/25/2008 6:27:29 PM
On Thu, 25 Sep 2008 11:27:29 -0700, Ken White <> wrote:

>> {quote:title=Guest wrote:}{quote}
>> Hello Everyone
>> 
>> For a long while I have used SAX for Pascal with a Delphi 6 project.
>> it is reading and writing very large XML files (over 800MB and
>> larger).  The project has been running for years - and I recently had
>> to upgrade to Delphi 2007 - and I discovered that SAX for Pascal has
>> not been ported to D2007.  The authors have provided full source -
>> which I have - but now I need to get it installed into D2007.  
>> Has anyone moved the Packages over to D2007?  
>> 
>> If this is not the best way of reading and writing really large XML
>> files anymore - any thoughts as to what is the best method?  
>> 
>
>Brad,
>
>I use OmniXML here. It works extremely well, is free with source, and the support for it by its' developers is fantastic. 
>
>You can find it at http://www.omnixml.com/
Ken,

Looks interesting.  However, is it just a parser?  Or can it be used
to write out large files as well?

What size files have you dealt with in the past?

Brad
0
G
9/26/2008 5:40:58 AM
> {quote:title=Guest wrote:}{quote}
> 
> Ken,
> 
> Looks interesting.  However, is it just a parser?  Or can it be used
> to write out large files as well?

It writes extremely well (and it's blazingly fast). It's also implemented 100% (AFAICT) in compliance with W3C standards for the DOM, so it's pretty source-compatible with everything else using the DOM model. (I ported from TurboPower's XMLPartner to OmniXML in a pair of relatively complex apps in less than an hour).

> What size files have you dealt with in the past?

I typically deal with files in the <1GB size range, but they're handled extremely quickly. (Everything is based on the GpHugeFile work of Primoz Gabrijelcic, and is based on a buffered TStream descendent, I believe, although I haven't dug into it that deeply myself.) Primoz is involved in OmniXML, and in fact was the one who provided the exceptional (and very fast) support with the only question I had; he even provided an actual code solution to the issue I was having. You can find that in the Support for
ums on the OmniXML site under the Help forum.

I'd suggest you just give it a try. As I said, it's standards-compliant, so it should be pretty easy to put some code together to test. (I can send you some sample code if you'd like, but the demos that come with OmniXML are really well done. For writing, look at the Xtreme demo; for parsing, the XPath demo is well done.)

Edited by: Ken White on Sep 26, 2008 11:22 AM - Fixed typo. Should have been < 1GB, not > 1GB.
0
Ken
9/26/2008 3:23:38 PM
On Fri, 26 Sep 2008 08:23:38 -0700, Ken White <> wrote:

>> {quote:title=Guest wrote:}{quote}
>> 
>> Ken,
>> 
>> Looks interesting.  However, is it just a parser?  Or can it be used
>> to write out large files as well?
>
>It writes extremely well (and it's blazingly fast). It's also implemented 100% (AFAICT) in compliance with W3C standards for the DOM, so it's pretty source-compatible with everything else using the DOM model. (I ported from TurboPower's XMLPartner to OmniXML in a pair of relatively complex apps in less than an hour).
>
>> What size files have you dealt with in the past?
>
>I typically deal with files in the <1GB size range, but they're handled extremely quickly. (Everything is based on the GpHugeFile work of Primoz Gabrijelcic, and is based on a buffered TStream descendent, I believe, although I haven't dug into it that deeply myself.) Primoz is involved in OmniXML, and in fact was the one who provided the exceptional (and very fast) support with the only question I had; he even provided an actual code solution to the issue I was having. You can find that in the Support fo
r
>ums on the OmniXML site under the Help forum.
>
>I'd suggest you just give it a try. As I said, it's standards-compliant, so it should be pretty easy to put some code together to test. (I can send you some sample code if you'd like, but the demos that come with OmniXML are really well done. For writing, look at the Xtreme demo; for parsing, the XPath demo is well done.)
>
>Edited by: Ken White on Sep 26, 2008 11:22 AM - Fixed typo. Should have been < 1GB, not > 1GB.
Ken,

thank you for your comments.  I will give it a try.

Bradley
0
G
9/26/2008 4:49:06 PM
You can try to use TALXmlDocument from alcinoe:
http://sourceforge.net/projects/alcinoe

it's work in Delphi 2007, support sax, and it's VERY VERY similar to 
XmlDocument (same property, same functions, simply replace TXmlDoc by 
TalXmlDoc to update your current project)
and very very must faster than TXmlDocument !

i use it everyday to parse xml file > 1GB !

stephane

Http://www.arkadia.com/fra/
Http://www.arkadia.com/usa/
Http://www.arkadia.com/rus/



"G. Bradley MacDonald" <brad_AT_timeacct_DOT_com> wrote in message 
news:20014@forums.codegear.com...
> Hello Everyone
>
> For a long while I have used SAX for Pascal with a Delphi 6 project.
> it is reading and writing very large XML files (over 800MB and
> larger).  The project has been running for years - and I recently had
> to upgrade to Delphi 2007 - and I discovered that SAX for Pascal has
> not been ported to D2007.  The authors have provided full source -
> which I have - but now I need to get it installed into D2007.
> Has anyone moved the Packages over to D2007?
>
> If this is not the best way of reading and writing really large XML
> files anymore - any thoughts as to what is the best method?
>
>
>
> The latest release of SAX for Pascal was on Ver 1.1 July 4, 2003
>
>
> G. Bradley MacDonald
> brad_AT_timeacct_DOT_Com
0
loki
10/5/2008 10:39:22 AM
You might want to look at vtd-xml, as its web site indicates, it seems well suited for processing large XML files

http://vtd-xml.sf.net





> {quote:title=Guest wrote:}{quote}
> Hello Everyone
> 
> For a long while I have used SAX for Pascal with a Delphi 6 project.
> it is reading and writing very large XML files (over 800MB and
> larger).  The project has been running for years - and I recently had
> to upgrade to Delphi 2007 - and I discovered that SAX for Pascal has
> not been ported to D2007.  The authors have provided full source -
> which I have - but now I need to get it installed into D2007.  
> Has anyone moved the Packages over to D2007?  
> 
> If this is not the best way of reading and writing really large XML
> files anymore - any thoughts as to what is the best method?  
> 
> 
> 
> The latest release of SAX for Pascal was on Ver 1.1 July 4, 2003
> 
> 
> G. Bradley MacDonald
> brad_AT_timeacct_DOT_Com

Edited by: rick h on Nov 21, 2009 2:28 PM

Edited by: rick h on Nov 21, 2009 2:29 PM

Edited by: rick h on Nov 21, 2009 2:31 PM
0
rick
11/21/2009 10:31:33 PM
Reply:

Similar Artilces:

Create an XML file from another XML file
I am using VS 2005 with C#.  I would like to create/generate an xml file from another XML with a differnt structure.  Basically, I want to create the actual xml file for my users whenever they raise my web service method. This is the actual structure of my xml i would like to create the following <Education>  <mydata>    <year>1988 - 1995</year>    <reward>High School Diploma</reward>    <school>Miami Secondary</school>  </mydata>  ...</Education> The original ...

XML Transform to 1 XML file to another XML File
Hello All  I am a novice programmer. I want to transform 1 xml file to another xml file. I am looking for a example code where user will select source xml file and create another xml file in different file with different fields with some default validation of parsing date and default values.  Can anyone help me or let me know any available source code in C#/ASP.Net?  Ms Disha I am assuming you know xslt, if not, you would have to go look at that.  But if you do, then this document will help you out http://www.csharpfriends.com/Articles/getArticle.aspx?articleID=...

XMl to XML
Hello,   I retrieved an XML from the dataset which retrieves data from the table in the database. I need to present the data  in a different structure. Is XSLT the way or are there any other options. please let me know. Thanks!! Yes XSLT is a good choice to convert xml file to other forms. You can take a look at XSLT Tutorial.Sincerely,Young Fang...

XML in, XML out
Hi All (and a happy holiday to those that will get a break), I am trying to read in an XML file of addresses. I need to remove all the address data from the file where code =~ /^000/ (there are none in the example data below). I need to reproduce that data 'as is', so I need to honour the tag structure, although the order of the tags doesn't need to be honoured. I have been trying to use XML::Simple and I had a go with XML::Smart but I haven't been able to get the results I want. My best effort is below (with XML::Simple). There are a couple of differences ...

Creating XML files with existing XML file format
My XML template file looks like this: <?xml version="1.0" standalone="yes"?> <txProductivityUpdate xmlns="RetailerGUI/txProductivityUpdate.xsd"> <header> <session-id>terchymppg2uieynky3pxfuz</session-id> <trancode>113</trancode> <request-timestamp>2003102012540750</request-timestamp> </header> <body> <chn-client>111</chn-client> <chn-customer>11111</chn-customer> <chn-name1>BERGEM, PHYLLIS</chn-name1>...

XML Parsing for large XML documents
I'm cross posting this question to perl4lib and xml4lib, hoping that someone will have a suggestion. I've created a very large (~54MB) XML document in RDF format for the purpose of importing related records into a database. Not only does the RDF document contain many thousands of individual records for electronic resources (web resources), but it also contains all of the "relationships" between those resources encoded in such a way that the document itself represents a rather large database of these resources. The relationships are multi-tiered. I've also...

create an xml schema from an xml file
Hello, I need to write an application that uses a xml file that i create and make a xml schema based on that file. I have a seperate company that is sending us the xml file and i want to get the info from the file and put it into a database. However, they need the my schema. I am reading up on how to use the DataSet.Writexml() and DataSet.WriteXmlSchema(). I know that there is some way of getting the data from the xml file using a DataSet, but i dont understand how. Has anyone done something similar to this that could help me at least get started? Thanks Dev Djinnx-Dev DjinnxWeb Develop...

Transforming XML into XML, creating a temporary file.
I've some XML which is formatted like such:1 <level1 name="category">2 <level2 name="subcategorya">3 <level3 name="item1" att="stuff" />4 </level2>5 <level2 name="subcategoryb">6 <level3 name="item2" att="stuff" />7 </level2>8 </level1>What I would like to do is transform it into something like this: 1 <category>2 <subcategorya>3 <item1 att="stuff" />4 </subc...

create XML object/read XML from file?
How do I create an XML object and load an XML file from the user's hard drive? I think creating the XML object goes like this: var xmlFile = Components.classes["@mozilla.org/xml/xml-document;1"].createInstance(Components.interfaces.nsIDOMXMLDocument); but I can't figure out how to populate the object with XML data from a file. Any suggestions? Thanks shinjin@gmail.com schreef: > How do I create an XML object and load an XML file from the user's hard > drive? > > I think creating the XML object goes like this: > > var xml...

Xml Schema from xml file
How should i create xml schema from xml file?  Do you mean an XSD?  That's easy--open the XML file in Visual Studio.  There will be a new XML menu button, and one option is to create an XSD.  If you have a lot of files, MS has a little tool at http://msdn.microsoft.com/en-us/vbasic/bb840042.aspx.If my post is your answer, please mark it as the answer. It will bring good karma.Crystal AllianceMy BlogFlorist Blogs  Thanks alot!!  It worked wonderfully...

XML content in a XML file
I have a requirement where I want to retrieve data from XML file.The concern is the data that has to be retrieved, it is again a XML data.e.g the following format is required<Gorups>   <Gorup>      <value>Some XML content</value>   </Gorup></Gorups>Thanks in advance Hello  Deepasmi, Are you facing problem in retrieval of specific xml node from xml file. Still problem is unclear.My Blog: DotNetError@BhatiaWorldDont forget to click "Mark as Answer" on the post that helped you.T...

Xml schema to Xml file
Is it possible to convert xml schema to xml file.How can i do read&writexmlschema using a dataset with xml schema. I converted a xml file to xml schema in a web appliction project.Then what should be location of shema that i need to give for a dataset I mean dataset.readxmlscheme(server.mappath(??))   Please let me know I am new to this field     dataset.ReadXml(Server.MapPath("xml\ProductList.xml")) will read the xml as well as the schema of thexml file. no need to convert it into schema file while using dataset as it will automatically define the ...

Load XMl file and read xml file
How do I load and read nodes from the xml file in c#.??? ThankYou Sample code: http://www.kirupa.com/net/reading_xml_directly_pg1.htmJack Yang.NET Developer Here is how you read xml file and loop and display nodesXmlDocument xDoc = new XmlDocument();xDoc.Load(xmlPath); foreach(XmlNode node in xDoc.SelectNodes("//nodeName you want to display")){Response.Write("name: " + node.Attributes["name"].InnerText + " value: " + node.Attributes["value"].InnerText);}...

How to create XML and XML with WebService
Hello...1) how to use XML and XML with Web Service with example.2) how to use read xml and write xml with coding with example.3) how to use read xml and write xml with dataset with example.thx,Tushar Poshiya       Everything about web-servicehttp://msdn2.microsoft.com/en-us/webservices/aa740685.aspx Ravi |   Share Knowledge   |   Online messengers   >>1) how to use XML and XML with Web Service with example. http://quickstarts.asp.net/QuickStartv20/webservices/doc/BPConformance.aspx (ASP.NET Web Services QuickS...

Web resources about - XML Creating and Parsing Very large XML Files - embarcadero.delphi.xml

Londonistan: How Britain is Creating a Terror State Within - Wikipedia, the free encyclopedia
The book encompasses a critique of multiculturalism , alleged weak policing, cultural relativism , and what Phillips calls a 'victim culture'. ...

Facebook’s Matt Kelly Describes Process Of Creating Plugin For WordPress
The plugin for WordPress launched by Facebook in June has been downloaded more than 180,000 times, and Engineer, Developer Relations Matt Kelly ...

Creating Online Ads that Drive Brand Awareness
New research shows how the creative elements of online ads impact brand awareness.

Bug - Creating a Video.Watches Action stopped working - Facebook-Entwickler
Since today executing a POST on a me/video.wacthes endpoints no longer works, it was working around 10 hours ago since then the call always returns ...

Facebook Creating Fourth-Party Privacy Rules for Facebook Connect Widget & Plugin Developers
In a recently-written post on the Facebook Developers Wiki, Facebook had added new rules for information sharing between websites that have implemented ...

The Obama campaign won by creating their own Facebook, with unique features for political campaigns.
... seen as too dangerous to nominate someone who is so thoroughly narcissistic, arrogant and dishonest. The Obama campaign won basically by creating ...

Creating A Great Startup Culture
... with any Bay Area startup these days (ping pong, beer on tap, catered lunches and Macbook Airs) – though these things are, of course, Creating ...

WIRED Creating the Nebula for iPad on the iTunes App Store
Get WIRED Creating the Nebula on the App Store. See screenshots and ratings, and read customer reviews.

Nets of freedom creating mesh networks 03.07.2013 - Flickr - Photo Sharing!
Explore Strelka Institute photo's photos on Flickr. Strelka Institute photo has uploaded 14526 photos to Flickr.

Creating the incorrigible Cartman - YouTube
"South Park" creators Trey Parker and Matt Stone were big fans of "All in the Family," a 1970s CBS sitcom. Perhaps this is why their most incorrigible ...

Resources last updated: 12/8/2015 2:45:07 AM