pre-ANNOUNCE: AnyData and DBD::AnyData

The module formerly known as DBD::RAM has been completely rewritten and
expanded and is now known as DBD::AnyData.  Please take a look at the
examples in the README attatched below to see some of its capabilities. 
Some of the many changes include

* Much more robust and flexible XML thanks to Michel
  Rodriguez's excellent XML::Twig

* Many new formats including HTML tables, Web logs,
  Passwd files, CSV, Fixed length files, etc.

* Either in-memory or file-based processing including file
  protections via flock() and record-at-a-time access

* An open API for other authors to create plugin format
  parsers either on-the-fly for quirky one time data or
  permanent to make the format available to other users

* A sister module that provides access to the same formats
  via multi-dimensional tied hashes

I'm giving the Pdx-mongers and the DBI listserv folks this advance copy
but am not CPANing it quite yet, mostly because the documentation sucks
and because I would like your input on the new interfaces before
finalizing them.

Read more about it and/or grab it at:

   http://www.vpservices.com/jeff/programs/AnyData/

Here's the README for AnyData.pm:

README FILE FOR AnyData

WHY USE IT?

  The AnyData suite of modules provides simple and uniform access
  to data from many sources -- perl arrays, local files, remote
  files retrievable via http or ftp -- and in many formats
  including flat files (CSV, Fixed Length, Tab Delimited, etc),
  standard format files (Web Logs, Passwd files, etc.),
  structured files (XML, HTML Tables) and binary files with
  parseable headers (mp3s, jpgs, pngs, etc).  All file access
  provides behind the scenes flocking and (in most cases) use of
  record-at-time access rather than pulling entire files into
  memory.

  There are two separate interfaces: 1) AnyData.pm supports
  searching and modifiying the data with multi-dimensional tied
  hashes and also the ability to convert to and from data in any
  of the supported formats; 2) DBD::AnyData provides a full DBI
  interface allowing use of DBI commands with the same subset of
  SQL currently provided by DBD::CSV and DBD::RAM.  DBD::AnyData
  replaces DBD::RAM which will no longer be supported.

  Here are a few examples of the tied hash interface, all also
  supported using standard DBI/SQL calls via DBD::AnyData:

  # FIND A USER'S HOME DIRECTORY IN A PASSWD FILE
  #
  my $users = adTie( 'Passwd', '/etc/passwd' );
  print $users->{jdoe}->{homedir};

  # DELETE A PLAYER FROM A PIPE DELIMITED GAMES DATABASE
  #
  my $players = adTie( 'Pipe', 'games.db', 'u' );
  delete $players->{jdoe};

  # RECURSIVELY LIST THE ARTISTS FOR ALL REGGAE MP3s
  # IN A SPECIFIED DIRECTORY TREE
  #
  my $music = adTie( 'Mp3', ['c:/My Music/'] );
  while ( my $song = each %$music ) {
      print $song->{artist},"\n" if $song->{genre} eq 'Reggae';
  }

  # RETRIEVE A CSV FILE FROM AN FTP SERVER
  # AND PRINT IT TO THE SCREEN AS AN HTML TABLE
  #
  # print adConvert( 'CSV', 'HTMLtable', 'ftp://foo.edu/pub/bar.csv' );

  # COUNT THE NUMBER OF HITS FOR A SPECIFIED PAGE IN A WEB LOG
  #
  my $hits = adTie( 'Weblog', 'access.log');
  print adCount( $hits , request => 'mypage.html' );

  # CREATE A CGI POP-UP MENU FROM A LISTING
  # OF THE VALUES OF A TABLE COLUMN
  #
  my $game = adTie( 'Pipe','games.db' );
  my @players  = adColumn( $game, 'player' );
  print CGI::popup_menu( 'players', \@players );

  # SELECT OR MODIFY MULTIPLE ROWS BASED ON COMPLEX CRITERIA
  # (this deletes all North American males over age 30)
  #
  my $data = adTie( 'Tab', 'mydb.tab');
  delete $data->{{ country => qr/us|mx|ca/,
                   gender  => 'eq m',
                   age     => '> 30',
                }};

WHAT ELSE DO I NEED?

  * Perl

  * The AnyData.pm module itself is pure Perl and does not
    depend on anything other than modules that come standard
    with Perl.  To use the remote ftp/http features, you must
    have the LWP bundle installed; to use the XML format, you
    must have XML::Parser and XML::Twig installed; to use the
    HTMLtable format for reading, you must have HTML::Parser
    and HTML::TableExtract installed but you can use the HTMLtable
    for writing with just the standard CGI module. To use DBI/SQL 
    commands, you must have DBI, SQL::Statement and DBD::File
    installed in addition to the AnyData modules.  All necessary
    modules are available on CPAN.  For windows users, all the 
    modules except AnyData, XML::Twig and HTML::TableExtract are
    also availble via ppm on www.activestate.com.

HOW DO I INSTALL IT?

  1.  Install Perl if not already installed

  2.  Unpack the compressed files.
      (AnyData-version.tar.gz or AnyData-version.zip)

  3a. If you are not familiar with the standard Perl
      makefile method, you can simply copy the files
  
  3b. If you are familiar with the standard Perl make 
      installation, just do as always (perl Makefile.PL; 
      make; make test; make install) this should also 
      work with dmake or nmake. 

HOW DO I USE IT?

  First you might like to try this simple script which 
  creates a database and inserts the string "hello new world"
  into a record  and then retrieves the record and prints it:

  #!perl -w
  use strict;
  use AnyData;
  my $table = adTie ('CSV','test.db','o',{cols=>'id,phrase'});
  $table->{1} = {phrase=>'hello new world'};
  print $table->{1}->{phrase}.

WHERE CAN I GET MORE INFO?

  After installing the module, type "perldoc AnyData" at 
  the command prompt, or just read the documentation at 
  the bottom of the AnyData.pm file.

WHO DUNNIT?

  Jeff Zucker <jeff@vpservices.com>

  Feel free to email me comments and suggestions, but please
  post questions requiring a response to the comp.lang.perl.modules
  newsgroup or (for DBD::AnyData only) via the dbi-users@perl.org
  listserv.

READ MORE AND GRAB THE MODULE AT

  http://www.vpservices.com/jeff/programs/AnyData/

Enjoy!

-- 
Jeff Zucker
0
jeff
2/6/2001 10:52:12 PM
perl.dbi.dev 1960 articles. 0 followers. Follow

2 Replies
517 Views

Similar Articles

[PageSpeed] 19

Jeff,
	Great work!  Two things I wondering about this:

	Is there a way to still load a database (using statement handle) to an in memory table?
	Is there a method to only "cache" X number of records before starting?

	I haven't had time to dig thru the source to determine either.

Keep up the good work!
Tom

On Tue, Feb 06, 2001 at 02:52:12PM -0800, Jeff Zucker wrote:
> The module formerly known as DBD::RAM has been completely rewritten and
> expanded and is now known as DBD::AnyData.  Please take a look at the
> examples in the README attatched below to see some of its capabilities. 
> Some of the many changes include
> 
> * Much more robust and flexible XML thanks to Michel
>   Rodriguez's excellent XML::Twig
> 
> * Many new formats including HTML tables, Web logs,
>   Passwd files, CSV, Fixed length files, etc.
> 
> * Either in-memory or file-based processing including file
>   protections via flock() and record-at-a-time access
> 
> * An open API for other authors to create plugin format
>   parsers either on-the-fly for quirky one time data or
>   permanent to make the format available to other users
> 
> * A sister module that provides access to the same formats
>   via multi-dimensional tied hashes
> 
> I'm giving the Pdx-mongers and the DBI listserv folks this advance copy
> but am not CPANing it quite yet, mostly because the documentation sucks
> and because I would like your input on the new interfaces before
> finalizing them.
> 
> Read more about it and/or grab it at:
> 
>    http://www.vpservices.com/jeff/programs/AnyData/
> 
> Here's the README for AnyData.pm:
> 
> README FILE FOR AnyData
> 
> WHY USE IT?
> 
>   The AnyData suite of modules provides simple and uniform access
>   to data from many sources -- perl arrays, local files, remote
>   files retrievable via http or ftp -- and in many formats
>   including flat files (CSV, Fixed Length, Tab Delimited, etc),
>   standard format files (Web Logs, Passwd files, etc.),
>   structured files (XML, HTML Tables) and binary files with
>   parseable headers (mp3s, jpgs, pngs, etc).  All file access
>   provides behind the scenes flocking and (in most cases) use of
>   record-at-time access rather than pulling entire files into
>   memory.
> 
>   There are two separate interfaces: 1) AnyData.pm supports
>   searching and modifiying the data with multi-dimensional tied
>   hashes and also the ability to convert to and from data in any
>   of the supported formats; 2) DBD::AnyData provides a full DBI
>   interface allowing use of DBI commands with the same subset of
>   SQL currently provided by DBD::CSV and DBD::RAM.  DBD::AnyData
>   replaces DBD::RAM which will no longer be supported.
> 
>   Here are a few examples of the tied hash interface, all also
>   supported using standard DBI/SQL calls via DBD::AnyData:
> 
>   # FIND A USER'S HOME DIRECTORY IN A PASSWD FILE
>   #
>   my $users = adTie( 'Passwd', '/etc/passwd' );
>   print $users->{jdoe}->{homedir};
> 
>   # DELETE A PLAYER FROM A PIPE DELIMITED GAMES DATABASE
>   #
>   my $players = adTie( 'Pipe', 'games.db', 'u' );
>   delete $players->{jdoe};
> 
>   # RECURSIVELY LIST THE ARTISTS FOR ALL REGGAE MP3s
>   # IN A SPECIFIED DIRECTORY TREE
>   #
>   my $music = adTie( 'Mp3', ['c:/My Music/'] );
>   while ( my $song = each %$music ) {
>       print $song->{artist},"\n" if $song->{genre} eq 'Reggae';
>   }
> 
>   # RETRIEVE A CSV FILE FROM AN FTP SERVER
>   # AND PRINT IT TO THE SCREEN AS AN HTML TABLE
>   #
>   # print adConvert( 'CSV', 'HTMLtable', 'ftp://foo.edu/pub/bar.csv' );
> 
>   # COUNT THE NUMBER OF HITS FOR A SPECIFIED PAGE IN A WEB LOG
>   #
>   my $hits = adTie( 'Weblog', 'access.log');
>   print adCount( $hits , request => 'mypage.html' );
> 
>   # CREATE A CGI POP-UP MENU FROM A LISTING
>   # OF THE VALUES OF A TABLE COLUMN
>   #
>   my $game = adTie( 'Pipe','games.db' );
>   my @players  = adColumn( $game, 'player' );
>   print CGI::popup_menu( 'players', \@players );
> 
>   # SELECT OR MODIFY MULTIPLE ROWS BASED ON COMPLEX CRITERIA
>   # (this deletes all North American males over age 30)
>   #
>   my $data = adTie( 'Tab', 'mydb.tab');
>   delete $data->{{ country => qr/us|mx|ca/,
>                    gender  => 'eq m',
>                    age     => '> 30',
>                 }};
> 
> WHAT ELSE DO I NEED?
> 
>   * Perl
> 
>   * The AnyData.pm module itself is pure Perl and does not
>     depend on anything other than modules that come standard
>     with Perl.  To use the remote ftp/http features, you must
>     have the LWP bundle installed; to use the XML format, you
>     must have XML::Parser and XML::Twig installed; to use the
>     HTMLtable format for reading, you must have HTML::Parser
>     and HTML::TableExtract installed but you can use the HTMLtable
>     for writing with just the standard CGI module. To use DBI/SQL 
>     commands, you must have DBI, SQL::Statement and DBD::File
>     installed in addition to the AnyData modules.  All necessary
>     modules are available on CPAN.  For windows users, all the 
>     modules except AnyData, XML::Twig and HTML::TableExtract are
>     also availble via ppm on www.activestate.com.
> 
> HOW DO I INSTALL IT?
> 
>   1.  Install Perl if not already installed
> 
>   2.  Unpack the compressed files.
>       (AnyData-version.tar.gz or AnyData-version.zip)
> 
>   3a. If you are not familiar with the standard Perl
>       makefile method, you can simply copy the files
>   
>   3b. If you are familiar with the standard Perl make 
>       installation, just do as always (perl Makefile.PL; 
>       make; make test; make install) this should also 
>       work with dmake or nmake. 
> 
> HOW DO I USE IT?
> 
>   First you might like to try this simple script which 
>   creates a database and inserts the string "hello new world"
>   into a record  and then retrieves the record and prints it:
> 
>   #!perl -w
>   use strict;
>   use AnyData;
>   my $table = adTie ('CSV','test.db','o',{cols=>'id,phrase'});
>   $table->{1} = {phrase=>'hello new world'};
>   print $table->{1}->{phrase}.
> 
> WHERE CAN I GET MORE INFO?
> 
>   After installing the module, type "perldoc AnyData" at 
>   the command prompt, or just read the documentation at 
>   the bottom of the AnyData.pm file.
> 
> WHO DUNNIT?
> 
>   Jeff Zucker <jeff@vpservices.com>
> 
>   Feel free to email me comments and suggestions, but please
>   post questions requiring a response to the comp.lang.perl.modules
>   newsgroup or (for DBD::AnyData only) via the dbi-users@perl.org
>   listserv.
> 
> READ MORE AND GRAB THE MODULE AT
> 
>   http://www.vpservices.com/jeff/programs/AnyData/
> 
> Enjoy!
> 

-- 
Thomas A. Lowery	stlowery@yahoo.com
http://tlowery.hypermart.net

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

0
stlowery
3/8/2001 8:43:33 PM
"Thomas A. Lowery" wrote:
> 
>         Great work! 

Thanks!

> Two things I wondering about this:
> 
>         Is there a way to still load a database (using statement handle) to an in memory table?
>         Is there a method to only "cache" X number of records before starting?

The answer is to both questions is yes there will be in the final
release but no there isn't in the current preview.  Hope to have the
final out in the next week or so.

> > Read more about it and/or grab it at:
> >
> >    http://www.vpservices.com/jeff/programs/AnyData/

-- 
Jeff
0
jeff
3/9/2001 5:28:58 PM
Reply: