Hello newsgroup

A test of the IdHMACSHA1 encryption method shows different results, and 
I am not sure if there is an error in my code or in the Indy code 
shipped with Delphi 2009.

The Delphi 2007 version works in an application which communicates with 
Amazon Web Services and so it seems to be the correct one.

Any suggestions are very welcome, maybe I just need to download a newer 
version of Indy?


program Test;

{$APPTYPE CONSOLE}

uses
   IdHMACSHA1, IdCoderMIME, IdGlobal, SysUtils;

function Base64Encode(const Input: string): string;
begin
   Result := TIdEncoderMIME.EncodeString(Input);
end;

function EncryptHMACSha1(Input, AKey: string): string;
begin
   with TIdHMACSHA1.Create do
   try
     Key := ToBytes(AKey);
     Result := BytesToString(HashValue(ToBytes(Input)));
   finally
     Free;
   end;
end;

begin

   WriteLn(Base64Encode(EncryptHMACSha1('Test Data', 
'D1AK1CzlmbUFGNd9Xh0KHNLGgKMEroMXQkHPMmaH')));
   ReadLn;

end.


Results
   in Delphi 2007: dJn4XDX4uY0LKL19ICSkHUfaEOI=
   in Delphi 2009: dBl4XDV4OQ0LKD19ICQkHUdaEGI=

-- 
Michael Justin
SCJP, SCJA
betasoft - Software for Delphi™ and for the Java™ platform
http://www.mikejustin.com - http://www.betabeans.de

Michael Justin wrote:
> Results
>    in Delphi 2007: dJn4XDX4uY0LKL19ICSkHUfaEOI=
>    in Delphi 2009: dBl4XDV4OQ0LKD19ICQkHUdaEGI=

You can tell by looking at these values that they differ only in some 
individual octets. This makes it likely that everything works exactly 
the same up to the point where the output bytes from the HashValue 
method are returned. The two result strings would have been completely 
different had the error occurred prior to the HashValue call.


 >      Result := BytesToString(HashValue(ToBytes(Input)));

The problem is in the IdGlobal.BytesToString call. The D2007 version of 
Indy10 will simply copy the bytes onto an AnsiString. The D2008 version 
defaults to using the 7-bit ASCII encoding, which probably is what is 
causing the issue.

"Henrick Hellström" <henrick@streamsec.se> wrote in message 
news:41360@forums.codegear.com...

> You can tell by looking at these values that they differ only in some
> individual octets. This makes it likely that everything works exactly
> the same up to the point where the output bytes from the HashValue
> method are returned. The two result strings would have been completely
> different had the error occurred prior to the HashValue call.

Does HashValue() return the same bytes in both versions, at least?  If not, 
then there is a bug in the underlying algorithm implementation.

-- 
Remy Lebeau (TeamB)

Remy Lebeau (TeamB) wrote:
> "Henrick Hellström" <henrick@streamsec.se> wrote in message 
> news:41360@forums.codegear.com...
> 
>> You can tell by looking at these values that they differ only in some
>> individual octets. This makes it likely that everything works exactly
>> the same up to the point where the output bytes from the HashValue
>> method are returned. The two result strings would have been completely
>> different had the error occurred prior to the HashValue call.
> 
> Does HashValue() return the same bytes in both versions, at least?  If not, 
> then there is a bug in the underlying algorithm implementation.

In this case, that is extremely unlikely. If the results Michael Justin 
see were in fact caused by a bug in TIdHMACSHA1.HashValue, it would have 
to be a bug put there on purpose by someone. I'm not saying that because 
I think the Indy crew are infallible (I know you aren't <g>), but 
because a typical accidental bug in either TIdSHA1 or TIdHMAC 
necessarily would propagate and cause a total discrepancy.

I go out on a limb and say it's a string encoding issue for sure. :)

Henrick Hellström wrote:

> The problem is in the IdGlobal.BytesToString call. The D2007 version of 
> Indy10 will simply copy the bytes onto an AnsiString. The D2008 version 
> defaults to using the 7-bit ASCII encoding, which probably is what is 
> causing the issue.

Many thanks for looking into this!

I have tried to reimplement the BytesToString and found that the problem 
actually is in the next step, the Base64 encoding with TIdEncoderMIME.

This code gives different results in Delphi 2007 and 2009 and I do not 
see an easy way to fix it:


WriteLn(TIdEncoderMIME.EncodeString(AnsiChar(116) + AnsiChar(153)));

Delphi 2007: dJk=
Delphi 2009: dCI=

Best Regards
-- 
Michael Justin
SCJP, SCJA
betasoft - Software for Delphi™ and for the Java™ platform
http://www.mikejustin.com - http://www.betabeans.de

Michael Justin wrote:
> I have tried to reimplement the BytesToString and found that the problem 
> actually is in the next step, the Base64 encoding with TIdEncoderMIME.

In such case the solution ought to be trivial:

function Base64Encode(const Input: TIdBytes): string;
begin
    Result := TIdEncoderMIME.EncodeBytes(Input);
end;

function EncryptHMACSha1(Input, AKey; string): TIdBytes;
begin
    with TIdHMACSHA1.Create do
    try
      Key := ToBytes(AKey);
      Result := HashValue(ToBytes(Input));
    finally
      Free;
    end;
end;

"Henrick Hellström" <henrick@streamsec.se> wrote in message 
news:41596@forums.codegear.com...

> In this case, that is extremely unlikely. If the results Michael Justin
> see were in fact caused by a bug in TIdHMACSHA1.HashValue,
> it would have to be a bug put there on purpose by someone. I'm
> not saying that because I think the Indy crew are infallible (I know
> you aren't <g>), but because a typical accidental bug in either
> TIdSHA1 or TIdHMAC necessarily would propagate and cause
> a total discrepancy.

I ask because I did make changes to the hash components' interfaces awhile 
back ago, and I don't know if they ever stablized yet.

-- 
Remy Lebeau (TeamB)

"Michael Justin" <michael.justin@nospam.gmx.net> wrote in message 
news:41723@forums.codegear.com...

> This code gives different results in Delphi 2007 and 2009

In 2009, EncodeString() takes a UnicodeString as input, not an AnsiString. 
AnsiString(116)+AnsiString(153) does not produce 
UnicodeString(116)+UnicodeString(153) when converted from Ansi to Unicode. 
It produces UnicodeString(116)+UnicodeString(8482) instead.  Ansi 153 is 
codepage-specific.  In codepage 1252 (Windows-1252, which most English 
systems use), it is the trademark character, but in other codepages it is a 
lead byte or undefined instead.  Ansi 153 is not a trademark character in 
Unicode (in fact, 153 is not a Unicode character at all - it is an undefined 
control character).  Unicode codepoint 8482 is the only official trademark 
character in Unicode.  When Ansi 153 in codepage 1252 is converted to UTF-16 
(which UnicodeString uses), it maps to codepoint 8482.  That conversion 
occurs before Indy sees the data, so this is not an Indy issue.

-- 
Remy Lebeau (TeamB)

Henrick Hellström wrote:
> Michael Justin wrote:
>> I have tried to reimplement the BytesToString and found that the problem 
>> actually is in the next step, the Base64 encoding with TIdEncoderMIME.
> 
> In such case the solution ought to be trivial:
> 
> function Base64Encode(const Input: TIdBytes): string;
> begin
>     Result := TIdEncoderMIME.EncodeBytes(Input);
> end;
> 
> function EncryptHMACSha1(Input, AKey; string): TIdBytes;
> begin
>     with TIdHMACSHA1.Create do
>     try
>       Key := ToBytes(AKey);
>       Result := HashValue(ToBytes(Input));
>     finally
>       Free;
>     end;
> end;

Excellent, this works! It seems to be a very new method, available in 
Delphi 2009 only. So I use {$IF RtlVersion < 20} to enable the old 
solution in versions belwo 2009. I will also perform some tests on 
Base64 encoding which is neccessary because the Amazon Simple Queue 
Service does not support UTF8 as message encoding (this is strange)...

Best Regards - thank you very much
-- 
Michael Justin
SCJP, SCJA
betasoft - Software for Delphi™ and for the Java™ platform
http://www.mikejustin.com - http://www.betabeans.de

Remy Lebeau (TeamB) wrote:
> I ask because I did make changes to the hash components' interfaces awhile 
> back ago, and I don't know if they ever stablized yet.

I appreciate that. Vice versa, I replied the way I did because the kind 
of instabilities that are likely to sneak in for such reasons will 
almost exclusively result in much greater discrepancies. With the 
exception of string encoding issues, it is extremely hard to get a HMAC 
to produce values that are *that* similar, unless you had done something 
really nasty deliberately, and I don't think you would. :)