delay loading DLLs for Mingw GCC Win32 perls

--------------000502020203070508060300
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I've had this code as a WIP for some years. Cleaned it up. Patches the 
poor implementation binutils devs (really one dev) did to make it. I 
almost want to rewrite all of the delay load infrastructure for GCC from 
scratch myself using perl+gas. This patch is for a CPAN module, but the 
code will have to be copy pasted into "win32/[GNU]Makefile[.mk]". I'm 
not sure how to write this code in a maintainable way. Since currently 
all I can think of is 2 copies of the 3 files mkdlylib.PL nulldlydescr.c 
setdlyhdr.PL in core repo. One for libperl.dll (p5p) and one for 
Win32.dll (cpan).

--------------000502020203070508060300
Content-Type: text/x-diff;
 name="0001-cpan-Win32-add-delay-loading-for-GCC-and-VC.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename*0="0001-cpan-Win32-add-delay-loading-for-GCC-and-VC.patch"

From 149b0612b1b27fb801ec49483f775ef7fac3a7ff Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Thu, 17 May 2018 01:35:27 -0400
Subject: [PATCH] cpan/Win32 add delay loading for GCC and VC

Most common use of Win32:: is cwd() and short vs long or rel vs abs path
functions. Followed by GetLastError. version.dll is very rarely used.
ole32.dll loads RPC service registration and COM and registry stuff
into the process. To speed up all perl modules EUMM building and later
testing. Put these 2 DLLs to be delay loaded. Visual C is easy to delay
load with. With GCC it is complicated, with little to no known public use
of the feature, and the feature is crudely hacked into LD without LD being
aware of what delay loaded DLLs are.

Technically, the linker and OS PE loader never need to be aware what delay
loaded DLLs are. Its just a pointer table initially all pointing at one
var arg, CPU context saving (think setjmp) function written in asm, that
loads the DLL, and writes the func ptr into the array, then longjmp()s
into the newly fetched function in another DLL. Subsequent calls goto the
real function and not the vararg loader one.

There is one serious bug with GCC delay loading, described at
https://sourceware.org/bugzilla/show_bug.cgi?id=14339 . I have worked
around that. And a spec violation that the delay loading structs arent
mentioned in the PE header. So static PE analysis tools dont show the
delayed imports for GCC, unlike for VC binaries where delayed imports are
listed by public PE tools. Delayed imports not being mentioned in the PE
header doesn't affect the delayed feature. But it prevents "binding" the
delayed imports to OS DLLs (perl doesn't currently do this, maybe one day
it will if I publish the code) and it prevents debugging tools from seeing
them.

To fix this second spec violation, compute the RVA of the delayed import
struct array and write it into the PE header after the linker puts the
DLL on disk. Use a GCC map file to find the abs addr of the struct inside
the DLL. Also add a null termination array slice "nulldlydescr.c" to stop
debugging tools from crashing or throwing errors, since VC always generates
the null filled struct. I named the symbol _NULL_DELAY_IMPORT_DESCRIPTOR
after how VC internally names it, but VC uses a different pattern of
symbol names than dlltool does.

dlltool is "_DELAY_IMPORT_DESCRIPTOR_ole32_dlya" VC is
"__DELAY_IMPORT_DESCRIPTOR_ole32"/"__NULL_DELAY_IMPORT_DESCRIPTOR"
---
 cpan/Win32/.gitignore     |   4 ++
 cpan/Win32/Makefile.PL    | 109 ++++++++++++++++++++++++++++++++
 cpan/Win32/Win32.xs       |  34 ++++++++++
 cpan/Win32/mkdlylib.PL    |  49 +++++++++++++++
 cpan/Win32/nulldlydescr.c |  13 ++++
 cpan/Win32/setdlyhdr.PL   | 157 ++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 366 insertions(+)
 create mode 100644 cpan/Win32/.gitignore
 create mode 100644 cpan/Win32/mkdlylib.PL
 create mode 100644 cpan/Win32/nulldlydescr.c
 create mode 100644 cpan/Win32/setdlyhdr.PL

diff --git a/cpan/Win32/.gitignore b/cpan/Win32/.gitignore
new file mode 100644
index 0000000..c16cbf7
--- /dev/null
+++ b/cpan/Win32/.gitignore
@@ -0,0 +1,4 @@
+*.tdef
+*.dlya
+*.s
+*.map
diff --git a/cpan/Win32/Makefile.PL b/cpan/Win32/Makefile.PL
index 0f16594..2ab68ee 100644
--- a/cpan/Win32/Makefile.PL
+++ b/cpan/Win32/Makefile.PL
@@ -2,17 +2,126 @@ use 5.006;
 use strict;
 use warnings;
 use ExtUtils::MakeMaker;
+use Config;
 
 unless ($^O eq "MSWin32" || $^O eq "cygwin") {
     die "OS unsupported\n";
 }
+#use delay loading for version.dll and ole32.dll
+#if true, checks will be performed if delay loading can be done
+#if checks fail, delay loading will not be done
+my $CanMSVCDelayLoad = 1;
+my $CanGCCDelayLoad = 1;
+
+#set this macro here, additional stuff might be appended
+my $OTHERLDFLAGS = '';
+
+CheckMSVCDelayLoad();
+CheckGCCDelayLoad();
 
 my %param = (
     NAME          => 'Win32',
     VERSION_FROM  => 'Win32.pm',
     INSTALLDIRS   => ($] >= 5.008004 && $] < 5.012 ? 'perl' : 'site'),
+    PL_FILES      => {},
+    DEFINE        => ($CanGCCDelayLoad ? '-DMY_GCC_DELAY_LOAD' : ''),
+    clean         => {FILES => '*.tdef *.dlya *.s *.map'},
+    dynamic_lib   => {
+        OTHERLDFLAGS =>
+            $OTHERLDFLAGS . ($CanMSVCDelayLoad ?
+                ' -DELAYLOAD:version.dll  -DELAYLOAD:ole32.dll delayimp.lib '
+                : $CanGCCDelayLoad ? ' -Wl,-Map=$(BASEEXT).map '
+                : '')
+    }
 );
 $param{NO_META} = 1 if eval "$ExtUtils::MakeMaker::VERSION" >= 6.10_03;
 $param{LIBS} = ['-L/lib/w32api -lole32 -lversion'] if $^O eq "cygwin";
 
 WriteMakefile(%param);
+
+sub CheckMSVCDelayLoad {
+    my $LocalCanMSVCDelayLoad = 0;
+    if($CanMSVCDelayLoad && $Config{ld} eq 'link') {
+        my $linkoutput = `link  /?`;
+        if($linkoutput =~ /delayload/i) {
+            $LocalCanMSVCDelayLoad = 1;
+        }
+    }
+    $CanMSVCDelayLoad = $LocalCanMSVCDelayLoad;
+    print "CanMSVCDelayLoad= $CanMSVCDelayLoad\n";
+}
+
+sub CheckGCCDelayLoad{
+    my $LocalCanGCCDelayLoad = 0; #cygwin not tested so dly ld disabled
+    if($CanGCCDelayLoad && $Config{cc} eq 'gcc' && $^O ne "cygwin") {
+        my $dlltoolhelp = `dlltool -h`;
+        my $nm = `nm -V`;
+        if(index($dlltoolhelp , 'output-delaylib') != -1
+           && index($nm, 'GNU General Public License') != -1) {
+            my $libpaths;
+            my $cmd = "$Config{cc} -print-search-dirs";
+            foreach(`$cmd`) {
+                if($_ =~ /^libraries: =(.+)$/) {
+                    $libpaths = $1;
+                    last;
+                }
+            }
+            die "command \"$cmd\" failed to return a library search path"
+                unless $libpaths;
+            my @libpaths = split(';', $libpaths);
+# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo
+# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth}
+# set wrong
+            my @mingwexarr = ExtUtils::Liblist->ext(
+                join(' ', map(qq["-L$_"], @libpaths)).' -lmingwex :nodefault',
+                1, 1
+            );
+            if($mingwexarr[0] =~ /mingwex/i){
+                my $runstr = 'nm -g "'.$mingwexarr[0].'"';
+                my $mingwexdump = `$runstr`;
+                if(index($mingwexdump,'__delayLoadHelper2') != -1){
+                    $LocalCanGCCDelayLoad = 1;
+                }
+            }
+        }
+    }
+    $CanGCCDelayLoad = $LocalCanGCCDelayLoad;
+    print "CanGCCDelayLoad= $CanGCCDelayLoad\n";
+}
+
+package MY;
+
+sub postamble {
+    return
+#note %.dlya is dmake only syntax
+($CanGCCDelayLoad ? '
+$(INST_DYNAMIC) : version.dlya ole32.dlya nulldlydescr.o
+
+%.dlya : mkdlylib.PL
+	$(PERLRUN) mkdlylib.PL '.$main::Config{cc}.' $*
+
+':'');
+
+}
+
+sub const_loadlibs {
+    my ($self) = @_;
+    if($CanGCCDelayLoad) {
+    #ordering of nulldlydescr.o is very important, must be after all .dlya files
+        $self->{LDLOADLIBS} .= ' version.dlya ole32.dlya nulldlydescr.o';
+        $self->{LDLOADLIBS} =~ s/-lversion//;
+        $self->{LDLOADLIBS} =~ s/-lole32//;
+    }
+    return $self->SUPER::const_loadlibs;
+}
+
+sub dynamic_lib {
+    my $self = shift; #note OTHERLDFLAGS is in @_
+    my $block = $self->SUPER::dynamic_lib(@_);
+    if($CanGCCDelayLoad) {
+        substr($block, -1, 1, '
+	$(PERLRUN) setdlyhdr.PL $@ $(BASEEXT).map
+');
+    }
+    return $block;
+}
diff --git a/cpan/Win32/Win32.xs b/cpan/Win32/Win32.xs
index de3764e..69abdcf 100644
--- a/cpan/Win32/Win32.xs
+++ b/cpan/Win32/Win32.xs
@@ -12,6 +12,40 @@
 #  define countof(array) (sizeof (array) / sizeof (*(array)))
 #endif
 
+#ifdef MY_GCC_DELAY_LOAD
+# include <delayimp.h>
+/*attribute dllimport causes access vio to integer 6 crash, the _imp_ is corrupt
+in the delay load libs dlltool generated, the mingw headers use attribute
+dllimport, get rid of attribute dllimport and the bug goes away
+
+see https://sourceware.org/bugzilla/show_bug.cgi?id=14339
+*/
+#  define GCC_VERSION (__GNUC__ * 10000 \
+                     + __GNUC_MINOR__ * 100 \
+                     + __GNUC_PATCHLEVEL__)
+/*pop push added in gcc 4.6*/
+#  if GCC_VERSION >= 40600
+#    pragma GCC diagnostic push
+#  endif
+#  pragma GCC diagnostic ignored "-Wattributes"
+extern HRESULT WINAPI CoCreateGuid(GUID * pguid);
+extern HRESULT WINAPI StringFromCLSID(const IID * const rclsid, LPOLESTR * lplpsz);
+extern void WINAPI CoTaskMemFree(LPVOID pv);
+#  if GCC_VERSION >= 40600
+#    pragma GCC diagnostic pop
+#  else
+#    pragma GCC diagnostic warning "-Wattributes"
+#  endif
+/* test offsets in setdlyhdr.pl */
+#  ifdef WIN64
+/* prob wrong */
+STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS64,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0);
+#  else
+STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS32,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0);
+#  endif
+STATIC_ASSERT_DECL(sizeof(ImgDelayDescr) == 0x20);
+#endif
+
 #define SE_SHUTDOWN_NAMEA   "SeShutdownPrivilege"
 
 #ifndef WC_NO_BEST_FIT_CHARS
diff --git a/cpan/Win32/mkdlylib.PL b/cpan/Win32/mkdlylib.PL
new file mode 100644
index 0000000..5953325
--- /dev/null
+++ b/cpan/Win32/mkdlylib.PL
@@ -0,0 +1,49 @@
+#!/usr/bin/perl -w
+use strict;
+require ExtUtils::Liblist;
+#file extension meanings, selected for easy makefile cleaning
+#tdef = temp def file
+#dlya = delay a file
+
+my $cc = $ARGV[0];
+my $basename = $ARGV[1];
+die "usage: perl mkdlylib.PL cc libbasename" unless $cc && $basename;
+my $dotaname = 'lib'.$basename.'.a';
+
+my $libpaths;
+my $cmd = "$cc -print-search-dirs";
+foreach(`$cmd`) {
+    if($_ =~ /^libraries: =(.+)$/) {
+        $libpaths = $1;
+        last;
+    }
+}
+die "command \"$cmd\" failed to return a library search path" unless $libpaths;
+my @libpaths = split(';', $libpaths);
+my @arr = ExtUtils::Liblist->ext(
+# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo
+# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth}
+# set wrong
+    join(' ', map(qq["-L$_"], @libpaths)).' -l'.$basename.' :nodefault'
+    , 0, 1
+);
+my $libpath;
+#remove the search loop, just 1 elem now
+foreach(@{$arr[4]}) {
+    if(index($_, $dotaname) != -1){
+        $libpath = $_;
+        last;
+    }
+}
+die("ExtUtils::Liblist can't find $dotaname") if ! defined($libpath);
+my $nmout =`nm -g $libpath`;
+my @exports = $nmout =~ m/^[[:xdigit:]]+ T _*(.+)$/mg;
+die "nm failed, output of nm was:\n\n".$nmout if !@exports;
+my $defstr = 'LIBRARY '.$basename.'.dll
+EXPORTS
+'.join("\n", @exports)."\n";
+open(FILE, '>', $basename.'.tdef') or die $!;
+print(FILE $defstr) or die $!;
+close(FILE) or die $!;
+my $retcode = system('dlltool -y '.$basename.'.dlya --input-def '.$basename.'.tdef');
+die "dlltool failed with exit code ".($retcode >> 8) if $retcode != 0;
diff --git a/cpan/Win32/nulldlydescr.c b/cpan/Win32/nulldlydescr.c
new file mode 100644
index 0000000..6264e82
--- /dev/null
+++ b/cpan/Win32/nulldlydescr.c
@@ -0,0 +1,13 @@
+/* Dont bother with headers, just make a zero filled ImgDelayDescr struct.
+ * This file is only used with GCC delay loading, to correct the array of
+ * ImgDelayDescr struct to meet Visual C linker/PE debugging tool standards
+ */
+char _NULL_DELAY_IMPORT_DESCRIPTOR [0x20]
+    __attribute__ ((section (".text$2")))
+/* GCC by default will create the object file section with 32 byte alignment
+   probably because this object is 32 bytes long but all ImgDelayDescr structs
+   created by dlltool have 4 byte align, and we dont want the linker to put
+   padding (aka uninit data) between ImgDelayDescr structs
+*/
+    __attribute__ ((aligned (4)))
+    = {};
diff --git a/cpan/Win32/setdlyhdr.PL b/cpan/Win32/setdlyhdr.PL
new file mode 100644
index 0000000..5d0dca8
--- /dev/null
+++ b/cpan/Win32/setdlyhdr.PL
@@ -0,0 +1,157 @@
+#!perl -w
+use strict;
+use Data::Dumper;
+#derived from Perl core win32/bin/exetype.pl
+
+# All the IMAGE_* structures are defined in the WINNT.H file
+# of the Microsoft Platform SDK.
+
+unless (0 < @ARGV && @ARGV < 3) {
+    print "Usage: $0 dllexefile mapfile\n";
+    exit 1;
+}
+
+my ($record,$magic,$signature,$offset,$va, $size, $win64);
+open EXE, '+<', $ARGV[0] or die "Cannot open $ARGV[0]: $!\n";
+binmode EXE;
+
+# read IMAGE_DOS_HEADER structure
+read EXE, $record, 64;
+($magic,$offset) = unpack "Sx58L", $record;
+
+die "$ARGV[0] is not an MSDOS executable file.\n"
+    unless $magic == 0x5a4d; # "MZ"
+
+# read signature, IMAGE_FILE_HEADER and first WORD of IMAGE_OPTIONAL_HEADER
+seek EXE, $offset, 0;
+read EXE, $record, 4+20+2;
+($signature,$size,$magic) = unpack "Lx16Sx2S", $record;
+
+die "PE header not found" unless $signature == 0x4550; # "PE\0\0"
+
+if($size == 224 && $magic == 0x10b) { # IMAGE_NT_OPTIONAL_HDR32_MAGIC
+    $win64 = 0;
+} elsif ($size == 240 && $magic == 0x20b) { # IMAGE_NT_OPTIONAL_HDR64_MAGIC
+    $win64 = 1;
+} else {
+    die "Optional header is neither in NT32 nor in NT64 format";
+}
+
+# Offset 0xE0 in the IMAGE_OPTIONAL_HEADER(32|64) is
+# OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT]
+seek EXE, $offset+0xE0, 0;
+read EXE, $record, 8;
+($va,$size) = unpack "LL", $record;
+if ($va == 0 && $size == 0) {
+    open MAP, '<', $ARGV[1] or die "Cannot open $ARGV[1]: $!\n";
+    binmode MAP;
+    {
+        $/ = undef;
+        my $map = <MAP>;
+        #baseaddr could also be extracted from PE header
+        $map =~ /__image_base__ = (0x[0-9a-f]+)/;
+        my $baseaddr = hex($1);
+        die "base address of PE file not found" unless $baseaddr;
+        my @descriptors = $map =~
+            / (0x[0-9a-f]+)                (?:_DELAY_IMPORT_DESCRIPTOR_|_NULL_DELAY_IMPORT_DESCRIPTOR)/g;
+        die "no delay loaded libraries found in $ARGV[1]" unless(@descriptors);
+        #TODO make sure all descriptors are 0x20 apart incase linker's design
+        #changes in future
+        $va = hex($descriptors[0])-$baseaddr;
+        $size = @descriptors*0x20
+    }
+    printf("adding IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT VA=0x%x Size=0x%x", $va, $size);
+    seek EXE, $offset+0xE0, 0;
+    print EXE pack "LL", $va, $size;
+} else {
+    die "Found existing delayed import header entry, not continuing";
+}
+close EXE;
+__END__
+
+Visual C crashes reading a dll without the NULL thunk. Theoretically Visual C
+should be using
+OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT].Size
+to iterate
+over the array of ImgDelayDescr structs but instead it uses "null termination".
+
+Other 3rd party PE tools like PE Explorer and Ida throw errors trying to read
+the x86 machine code as RVAs/pointers after these ImgDelayDescr structs, so null
+termination instead of .Size member seems to the universal implementation for
+parsing delay import descriptors. The ImgDelayDescr structs are allocated in
+.text section because of dlltool's/ld's implementation (or lack thereof) of
+delay loading which explains why there is machine code after the structs.
+
+C:\perl\src\win32> dumpbin /imports ..\lib\auto\win32\Win32.dll
+Microsoft (R) COFF/PE Dumper Version 7.10.6030
+Copyright (C) Microsoft Corporation.  All rights reserved.
+
+
+Dump of file ..\lib\auto\win32\Win32.dll
+
+File Type: DLL
+
+  Section contains the following delay load imports:
+
+    version.dll
+              00000001 Characteristics
+              70A49008 Address of HMODULE
+              70A50514 Import Address Table
+              70A502C8 Import Name Table
+              00000000 Bound Import Name Table
+              00000000 Unload Import Name Table
+                     0 time date stamp
+
+          70A47586               0 GetFileVersionInfoA
+          70A47576               1 GetFileVersionInfoSizeA
+          70A47566               A VerQueryValueA
+
+    ole32.dll
+              00000001 Characteristics
+              70A4900C Address of HMODULE
+              70A50500 Import Address Table
+              70A502B4 Import Name Table
+              00000000 Bound Import Name Table
+              00000000 Unload Import Name Table
+                     0 time date stamp
+
+          70A475CA               F CoCreateGuid
+          70A475BA              6A CoTaskMemFree
+          70A475AA             13E StringFromCLSID
+
+    (null)
+              A11CEC83 Characteristics
+              D0C804C7 Address of HMODULE
+              FA14A4A0 Import Address Table
+             16FA82444 Import Name Table
+             115A6E815 Bound Import Name Table
+             101349070 Unload Import Name Table
+              90669066 time date stamp
+
+
+DUMPBIN : fatal error LNK1000: Internal error during DumpDelayLoadImports
+
+  Version 7.10.6030
+
+  ExceptionCode            = C0000005
+  ExceptionFlags           = 00000000
+  ExceptionAddress         = 0043B42A (00400000) "C:\Program Files\Microsoft Vis
+ual Studio .NET 2003\Vc7\bin\link.exe"
+  NumberParameters         = 00000002
+  ExceptionInformation[ 0] = 00000000
+  ExceptionInformation[ 1] = 7EFDE044
+
+CONTEXT:
+  Eax    = FF03E044  Esp    = 0012E6E0
+  Ebx    = FF03E044  Ebp    = 0012E7BC
+  Ecx    = 0000DC00  Esi    = 011A5AC8
+  Edx    = 7FFA0000  Edi    = 00000000
+  Eip    = 0043B42A  EFlags = 00010286
+  SegCs  = 0000001B  SegDs  = 00000023
+  SegSs  = 00000023  SegEs  = 00000023
+  SegFs  = 0000003B  SegGs  = 00000000
+  Dr0    = 0012E6E0  Dr3    = FF03E044
+  Dr1    = 0012E7BC  Dr6    = 0000DC00
+  Dr2    = 00000000  Dr7    = 00000000
+
+C:\perl\src\win32>
-- 
2.5.0.windows.1


--------------000502020203070508060300--
0
bulk88
5/17/2018 5:42:23 AM
perl.perl5.porters 47182 articles. 0 followers. Follow

0 Replies
5 Views

Similar Articles

[PageSpeed] 47

Reply: