Parse a Remote Folder to get newly created files

My scenario:
An external application is creating a xml file every 5-10 seconds in a 
file server.
I need to parse each file, just once, to create a record in a SQL table 
on real time.
I tried some components (from LMD's and JAM's, see my post in 
thirdpartytools.general) but they fail on notifing the name of the file 
created on a remote PC, just report a folder is updated.
Since at the end of the day there are about than 10,000 files in the 
folder I don't want to parse every single file, just the last (newest), 
to check if it's already on the table...
Any help will be appreciated.
Best Regards
Francisco Alvarado
0
Francisco
4/22/2015 2:51:16 PM
embarcadero.delphi.nativeapi 1236 articles. 1 followers. Follow

8 Replies
1037 Views

Similar Articles

[PageSpeed] 5
Get it on Google Play
Get it on Apple App Store

Francisco Alvarado wrote:

> My scenario:
> An external application is creating a xml file every 5-10 seconds in
> a file server.
> I need to parse each file, just once, to create a record in a SQL
> table on real time.
> I tried some components (from LMD's and JAM's, see my post in 
> thirdpartytools.general) but they fail on notifing the name of the
> file created on a remote PC, just report a folder is updated.
> Since at the end of the day there are about than 10,000 files in the 
> folder I don't want to parse every single file, just the last
> (newest), to check if it's already on the table...
> Any help will be appreciated.
> Best Regards
> Francisco Alvarado

I don't see a way around using a FindFirst/FindNext loop to scan the
files and check their dates. Can you move the scanned files to a
subfolder after you are done with them? That would cut down on the
number of files you have to scan on each round.

-- 
Peter Below (TeamB)
0
Peter
4/22/2015 6:13:10 PM
On 04/22/2015 12:13 PM, Peter Below wrote:
> I don't see a way around using a FindFirst/FindNext loop to scan the
> files and check their dates. Can you move the scanned files to a
> subfolder after you are done with them? That would cut down on the
> number of files you have to scan on each round.
>
Peter, I can't do that, the folder is there for many others to check logs.
Best Regards
Francisco Alvarado
0
Francisco
4/22/2015 9:25:42 PM
On 04/22/2015 12:13 PM, Peter Below wrote:
> I don't see a way around using a FindFirst/FindNext loop to scan the
> files and check their dates. Can you move the scanned files to a
> subfolder after you are done with them? That would cut down on the
> number of files you have to scan on each round.
>
If I use FindFirst/FindNext, which will be the first file found?
Is there a way to be sure it will the most recent or the oldest?
Just wondering, because the file names will be like this:
Order_10.225.109.9_17c9c028-7e6a-48b5-9997-252cb74363ed.xml
Order_10.225.109.9_17 will not change and the rest are random characters.
Best Regards
Francisco Alvarado
0
Francisco
4/22/2015 9:40:30 PM
On 04/22/2015 7:55 PM, Adem Meda wrote:
> Why not write another small app (a Windows Service) that runs on the file
> server that does all this on there --and communicates with your remote app
> through TCP/IP or whatever.
>
Thank you Adem, but I can't run anything on the server side.
I'm going to test with FindFirst/FindNext loop and see how it works at 
the end of the day, when the number of files is high.
Right now I'm searching for examples...
Best Regards
Francisco Alvarado
0
Francisco
4/23/2015 1:01:01 AM
Francisco Alvarado wrote:

> My scenario:
> An external application is creating a xml file every 5-10 seconds in a 
> file server.
> I need to parse each file, just once, to create a record in a SQL table 
> on real time.

Why not write another small app (a Windows Service) that runs on the file
server that does all this on there --and communicates with your remote app
through TCP/IP or whatever.
0
Adem
4/23/2015 1:55:31 AM
On 04/22/2015 8:51 AM, Francisco Alvarado wrote:
> My scenario:
> An external application is creating a xml file every 5-10 seconds in a
> file server...

I'm using FindFirst/FindNext loop to scan the remote folder and a 
TDateTime variable to check if the file timestamp is newer. If it is 
then I process it and at the end of the loop I update the variable.
I think it may reduce the number of files to be inserted in the database.
Best Regards
Francisco Alvarado
0
Francisco
4/23/2015 5:13:34 AM
Francisco Alvarado wrote:

> On 04/22/2015 12:13 PM, Peter Below wrote:
> > I don't see a way around using a FindFirst/FindNext loop to scan the
> > files and check their dates. Can you move the scanned files to a
> > subfolder after you are done with them? That would cut down on the
> > number of files you have to scan on each round.
> > 
> If I use FindFirst/FindNext, which will be the first file found?
> Is there a way to be sure it will the most recent or the oldest?

No, there is no guaranteed sequence as far as I know. But that may
depend on the server OS, so why don't you run some experiments? Log the
filenames to a local file for several scans and compare the sequence...



-- 
Peter Below (TeamB)
0
Peter
4/23/2015 8:38:41 AM
On 04/22/2015 8:51 AM, Francisco Alvarado wrote:
> Any help will be appreciated.
>
I just ended using FindFirst/FindNext loop and it works fine so far...
Thank you all for your help.
Best Regards
Francisco Alvarado
0
Francisco
4/24/2015 3:35:40 AM
Reply:

Similar Artilces:

Read a dynamically created html file and re-write it into a newly created file
I need to read a dynamically created html file and save the contents to a new file in a directory. The point is, I have an application that takes imput from the user, they upload files into my database.  I then display a dynamically created page with the files that they have uploaded grouped by week.  I have the preview of their file showing, I cannot figure out how to get that file to save without using the menu's save button.  This option doesn't work becuase I have to save the file, not the user.  Any ideas would be appreciated.   Thank you, I do no...

Newly created local folders are getting duplicated
I have begun to notice that every time I create a local folder, to hold old email messages, the next time I go into that folder there is a second copy, exactly the same as the original with the same contents. Anyone seen this before? ...

Is there a way to get the created date and content of a remote file?
Normally , I use a FileUpload control to upload file and get the file content using FileUpload1.PostedFile.InputStream, but I don't know how to get the created date of the uploaded file. If the file located in remote server, such as http://www.diychart.com/1.txt, how can I get the created date and content of a remote file? thank!SuperCool Multiple ZIP - A utility to unzip multiple files and work with multiple zip filesSuperCool Random Number Generator I hope you can get the file creation date thru Javascript. Write a javascript on the upload button OnClick event and get the created da...

How to Create New Folders To Store Newly Uploaded Files
Can anyone give me an idea of how I go about Creating a new folder and storing the uploaded files in this folder, each time a user uses the upload facility?   System.IO.Directory.CreateDirectory(Server.MapPath("newfolder")); A fine is a tax for doing wrong. A tax is a fine for doing well.__________________________________________________Please remember to click “Mark as Answer” on the post that helps you.  To go about creating files/folders in the virtual directory you can use classes/methods in the System.IO namespace. However, you still need privilidges to...

How to create solution file at the same folder as project file?
Every time after I create new project file, solution file will create into C:\Documents and Settings\user\documents\... How to create solution file at the same folder as project file? All that you need to do is move the sln file and suo file out of this directory into the one you would like for it to be located and then double click on the sln file it will update the paths. Also if you do not like the path you can go into options -> environment -> projects and solutions and change the path. To my knowledge if you want every project you create to store the solution files in the folder...

how to get date created of file while uploading any file
Hi, I am using FileUpload control in C# to upload files into database in my web application. My requirement is that I need to have Date Created, Date Modified and Last Access date of the file being uploaded and store the same into my database. I used "System.IO.File.GetCreationTime(finfo.FileName).ToShortDateString();" which is working fine in my local system. But after deploying on my webserver, it is giving me only "12/31/1600". Does any one know any work around this? Regards,Krunal hi doshi_krunaln, If you ar eusing ToShortDateString(0 function it will always re...

Limit of 32767 files in Win'98 folder when creating GW 6.0.3 remote database?
I'm trying to finish the creation of a remote database and seem to have hit a hitch whereby the ROFDATA folder has 32767 files, and it appears that it won't take anymore (weird errors when running send/retrieve, GWCHECK won't do a structural rebuild, I can't rename NGWCHECK.DB, ...). Is this a limitation under Win'98 (FAT32)? Why is GW creating so many files, does each one represent an email? Is this the max on a remote database, 32767 files? Thanks James 32767 is a magic number, so yes, I'd say that's a max. -- Michael J. Bell No...

I have an .aspx file located in some 'a' folder within my web folder. How can i get the path of the web folder in....
I have an .aspx file located in some 'a' folder within my web folder. How can i get the path of the web folder in order to take the path and post it in the NavigateUrl Property of a asp:Hyperlink. the problem starts when i'm clicking on the hyperlink(that located in my master page). when i'm clicking from 'a' folder it using the 'a' folder as root folder and he  searching the file from a diffrent path. (in a none exist path) for example: the navigate url is: "columns/column1.aspx" and from 'a' folder (it's a diffrent...

How to create a folder under project name and then upload a file into this folder?
How to create a folder under project name and then use FileUpload to upload a file into this folder? manually create a folder under the root application folder.  u upload a file into the newly folder by FileUpload.PostedFile.SaveAs(Server.MapPath("/NEWFOLDERNAME")); Happy Coding  Haissam Abdul MalakMCAD.NET| Blog | hi, just add a new folder to the root of your project through solution explorer, say named myfolder and then in the code behind  protected void Button1_Click(object sender, EventArgs e)    {      &nbs...

Prevent user creating folders but allow creat files
I have an issue whereby i am trying to re-structure the shared folders on the data volume of our netware 6.5 server. As it currently stands users are obtaining their rights from container level all the way down. The current data folder structure has a large amount of levels of sub-folders, i have tried to insist that this should be allowed but am ignored. So the users are able to see from the top level folder all the way down, i am trying to implement the use of groups where by the users would obtain the rights to that specific folder - keeping it consistent with names being th...

being able to create folders on a sahred folder/remote server
Im supposed to develop a feature in my asp application which allows a user to create a folder. This folder is created on a remote server/shared drive. Also this ability to create folder is not available to all only those in the admin roles. can someone help me with this feature.     since its on a remote server, you will have to impersonate a network user who has appropriate privileges  to create the folder  See this article to see how you impersonate a user http://support.microsoft.com/kb/306158 Since you have roles set up .You first ...

Prevent user creating folders but allow create files
I have an issue whereby i am trying to re-structure the shared folders on the data volume of our netware 6.5 server. As it currently stands users are obtaining their rights from container level all the way down. The current data folder structure has a large amount of levels of sub-folders, i have tried to insist that this should be allowed but am ignored. So the users are able to see from the top level folder all the way down, i am trying to implement the use of groups where by the users would obtain the rights to that specific folder - keeping it consistent with names being th...

How to create file folder and back up file automatically everyday?
I would like to write a program which can back up file automatically everyday. we will have many file folder for reports is it possible to back up file by create a new file folder under every file folder of reports and then copy report to the new create file folder as a back file?   this idea looks very diffcult and seems can't not be finished.. thank you  Use windows service http://msdn.microsoft.com/msdnmag/issues/05/03/SchedulingASPNETCode/default.aspx or use multithreading http://www.beansoftware.com/ASP.NET-Tutorials/Multithreading-Thread-Pool.aspx Ravi | &nbs...

Trying to get File.open to open file on remote server
The title is kind of lame. What I'm doing is that I had been developing on the production server (don't ask), and now I've moved development to my workstation. I have a bit of code in an aspx the opens a pre-formatted html and puts it into a label. The code is as follows. Dim FILENAME As String = "D:\Inetpub\wwwroot\articles\news.htm" Dim objStreamReader As StreamReader objStreamReader = File.OpenText(FILENAME) articles.Text = objStreamReader.ReadToEnd() objStreamReader.Close() This worked fine and the aspx on the product...

Web resources about - Parse a Remote Folder to get newly created files - embarcadero.delphi.nativeapi

Runs created - Wikipedia, the free encyclopedia
With regard to an offensive player, the first key question is how many runs have resulted from what he has done with the bat and on the basepaths. ...

Lexus Promotes 2014 Lexus IS With #LexusInstafilm, Created With Photos By Instagram Users
... in Angel Stadium in Anaheim, Calif., to promote the 2014 Lexus IS in a unique way by creating what it called “the first collaboratively created ...

I am Colin's mom, I created this page... - Happy Birthday Colin - Facebook
I am Colin's mom, I created this page for my amazing, wonderful, challenging son who is about to turn 11 on March 9th. Because of Colin's disabilities,... ...

Facebook allows targeting on page-created events
... to have a ladies’ night event, they could properly target so only women could see the event page. This capability is only for events created ...

PDF (file format) : How was the PDF format created?
Answer (1 of 6): I was there for the whole thing. It was the 90's and Adobe was doing well. In addition to the Systems department which handled ...

DrawTo - Send and receive drawings seeing as they are created on the App Store
Read reviews, compare customer ratings, see screenshots, and learn more about DrawTo - Send and receive drawings seeing as they are created. ...

The iconic statue of a knotted gun barrel outside U.N. headquarters was created by Swedish artist Fredrik ...
Rights Groups Warn Against Diluted Arms Trade Treaty By Tressia Boukhors UNITED NATIONS, Feb 18, 2012 (IPS) - After a week of tense negotiations, ...

Bad News Barrett Interview: Cody Rhodes created my gimmick, will BNB win the WWE title? more - YouTube ...
twitter.com/ChrisVanVliet 3-time Emmy winning Entertainment Reporter Chris Van Vliet from CBS-19 chats with Bad News Barrett in Cleveland. He ...

IMF warns GCC countries of $175 billion hole created by falling oil prices
The IMF believes that lower oil prices could knock nearly 1 percentage point off economic growth rates in GCC countries.

This is probably the world’s largest billboard created with MS Paint
The billboard, advertising Construct 2 game creation platform, was made with MS Paint. The billboard is by South Bermondsey station in south-east ...

Resources last updated: 12/30/2015 8:29:28 PM