A malformed message causing issue with the decoder [Edit]

I have another malformed message causing issue with Indy decoder:

{code}
--KEJ3581-rC-4502470-254=_!
Content-Type: text/html; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64

\CjxodG1sPgo8Y2VudGVyPgo8Ym9keT4KPCEtLSAyaW5kaXQgLSBpbiBnbHVtYSAtIHNhIGltcHJ1
bXV0IGNoaXB1bCB1bnVpIGNvbnNpbGllciAtLT48IS0tICwsQSwgbWkgc2UgcGFyZSBjYSBtaS1h
{code}

If I remove the "\" which is at the beginning of the MIME part the message is decoded well. "\" is not in the list of valid characters for base64.

Outlook Express, Outlook and other programs correctly decode the message, most likely because they ignore the invalid characters and Indy likely takes these invalid characters into account instead of ignoring them.

I think the problem is that people who send these messages test them on Outlook - they see it works and conclude it is good. Problem is that Indy decoder (not encoder which should be strict) is not tolerant enough of such malformed data and might be further adjusted to make it more compatible. I understand that Indy follows standards and that's fine for encoder. But decoder should have certain degree of tolerance against such malformed messages. I have also seen evidence of this being the case because in 
the MIME decoder of the Ararat Synapse decoder they have the following line:

{code}
//was 7bit before, but this is more compatible with RFC-ignorant outlook
Encoding := '8BIT'; 
{code}

It is clear from above evidence that they use 8bit default encoding as it seems to decode more messages properly (this is also a problem for Indy defaults), that many people test their web or php messages against Outlook as a "reference" of being well formed and as Outlook has a tolerance against such messages in its own decoder the end result is that decoders which respect standards need to be tweaked to be more compatible with such malformed messages.

Is it possible to strip out all non-valid data from the input (backslash "\" character in this case) to make the base64 input valid like Outlook and other programs do at the moment and make the decoder more compatible? In other words, if base64 decoder encounters an invalid character it just skips ahead as if the invalid character is not there? That should make the decoder happy and should not break the existing Indy stuff.
0
John
1/19/2015 4:00:44 PM
embarcadero.cppbuilder.socket 566 articles. 0 followers. Follow

2 Replies
1103 Views

Similar Articles

[PageSpeed] 22

John wrote:

{code}
\CjxodG1sPgo8Y2VudGVyPgo8Ym9keT4KPCEtLSAyaW5kaXQgLSBpbiBnbHVtYSAtIHNhIGltcHJ1
bXV0IGNoaXB1bCB1bnVpIGNvbnNpbGllciAtLT48IS0tICwsQSwgbWkgc2UgcGFyZSBjYSBtaS1h
{code}

Without seeing the complete email, I can only guess, but I suspect the '\' 
is actually not part of the base64 data itself, but it part of a higher level 
transfer encoding that was not decoded during reading.  With the '\' present, 
the base64 data shown would be 153 characters, which is not an even multiple 
of 4 that base64 requires, and it also makes the first line be 77 characters 
long.  Both of those are against standard base64 practice.  With the '\' 
removed, the first line is 76 characters long, and the base64 data is 152 
characters.  Both of which are within standard base64 ranges.

> Outlook Express, Outlook and other programs correctly decode the
> message, most likely because they ignore the invalid characters and
> Indy likely takes these invalid characters into account instead of
> ignoring them.

Yes, it does.  So I re-read RFC 2045, which defines MIME's base64 algorithm. 
 It specifically says that illegal characters not defined in the base64 alphabet 
must be ignored during decoding.  So be it, but easier said than done, since 
Indy's base64 decoder shared common logic with the 00E, UUE, and XXE decoders. 
 So I would need to make sure not to break them while fixing the base64 decoder.

> Is it possible to strip out all non-valid data from the input (backslash 
"\"
> character in this case) to make the base64 input valid like Outlook and
> other programs do at the moment and make the decoder more compatible?

Indy uses streaming decoding, so the data may not be available up front when 
decoding begins, and thus cannot be stripped out ahead of time.  But I will 
look into having the base64 decoder ignore illegal data that is encountered 
during its decoding.

-- 
Remy Lebeau (TeamB)
0
Remy
1/19/2015 10:23:59 PM
> {quote:title=Remy Lebeau (TeamB) wrote:}{quote}
> Without seeing the complete email, I can only guess, but I suspect the '\' 
> is actually not part of the base64 data itself, but it part of a higher level 
> transfer encoding that was not decoded during reading.  With the '\' present, 
> the base64 data shown would be 153 characters, which is not an even multiple 
> of 4 that base64 requires, and it also makes the first line be 77 characters 
> long.  Both of those are against standard base64 practice.  With the '\' 
> removed, the first line is 76 characters long, and the base64 data is 152 
> characters.  Both of which are within standard base64 ranges.

Thanks for looking into this. I PM-ed you a full message if you need it just in case. There is nothing extra in it. \ just appears out of place and should not be there.
0
John
1/19/2015 11:01:16 PM
Reply:

Similar Artilces:

A malformed message causing issue with the decoder [Edit] #2
I have another malformed message causing issue with Indy decoder: {code} --KEJ3581-rC-4502470-254=_! Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 \CjxodG1sPgo8Y2VudGVyPgo8Ym9keT4KPCEtLSAyaW5kaXQgLSBpbiBnbHVtYSAtIHNhIGltcHJ1 bXV0IGNoaXB1bCB1bnVpIGNvbnNpbGllciAtLT48IS0tICwsQSwgbWkgc2UgcGFyZSBjYSBtaS1h {code} If I remove the "\" which is at the beginning of the MIME part the message is decoded well. "\" is not in the list of valid characters for base64. Outlook Express, Outlook and other programs corre...

A malformed message causing issue with the decoder [Edit] #2
I have another malformed message causing issue with Indy decoder: {code} --KEJ3581-rC-4502470-254=_! Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 \CjxodG1sPgo8Y2VudGVyPgo8Ym9keT4KPCEtLSAyaW5kaXQgLSBpbiBnbHVtYSAtIHNhIGltcHJ1 bXV0IGNoaXB1bCB1bnVpIGNvbnNpbGllciAtLT48IS0tICwsQSwgbWkgc2UgcGFyZSBjYSBtaS1h {code} If I remove the "\" which is at the beginning of the MIME part the message is decoded well. "\" is not in the list of valid characters for base64. Outlook Express, Outlook and other programs corre...

A malformed message causing issue with the decoder [Edit] #2
I have another malformed message causing issue with Indy decoder: {code} --KEJ3581-rC-4502470-254=_! Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 \CjxodG1sPgo8Y2VudGVyPgo8Ym9keT4KPCEtLSAyaW5kaXQgLSBpbiBnbHVtYSAtIHNhIGltcHJ1 bXV0IGNoaXB1bCB1bnVpIGNvbnNpbGllciAtLT48IS0tICwsQSwgbWkgc2UgcGFyZSBjYSBtaS1h {code} If I remove the "\" which is at the beginning of the MIME part the message is decoded well. "\" is not in the list of valid characters for base64. Outlook Express, Outlook and other programs corre...

A malformed message causing issue with the decoder
I have another malformed message causing issue with Indy decoder: {code} --KEJ3581-rC-4502470-254=_! Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 \CjxodG1sPgo8Y2VudGVyPgo8Ym9keT4KPCEtLSAyaW5kaXQgLSBpbiBnbHVtYSAtIHNhIGltcHJ1 bXV0IGNoaXB1bCB1bnVpIGNvbnNpbGllciAtLT48IS0tICwsQSwgbWkgc2UgcGFyZSBjYSBtaS1h {code} If I remove the "\" which is at the beginning of the MIME part the message is decoded well. "\" is not in the list of valid characters for base64. Outlook Express, Outlook and other programs corre...

A malformed message causing issue with the decoder #2
I have another malformed message causing issue with Indy decoder: {code} --KEJ3581-rC-4502470-254=_! Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 \CjxodG1sPgo8Y2VudGVyPgo8Ym9keT4KPCEtLSAyaW5kaXQgLSBpbiBnbHVtYSAtIHNhIGltcHJ1 bXV0IGNoaXB1bCB1bnVpIGNvbnNpbGllciAtLT48IS0tICwsQSwgbWkgc2UgcGFyZSBjYSBtaS1h {code} If I remove the "\" which is at the beginning of the MIME part the message is decoded well. "\" is not in the list of valid characters for base64. Outlook Express, Outlook and other programs corre...

a message causing Indy mesasage decoder to enter infinite loop [Edit]
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. I have Indy 5231 which is not the latest but it is possibly it is a problem not addressed yet. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF...

a message causing Indy mesasage decoder to enter infinite loop [Edit]
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF-8?B?RDEw?= =?UTF-8?B?MDAgTm8=?= =?UTF-8?B?dw==?= From: =?UTF-8?B?TmU=?= =?UTF-8?B?eHRQ?= =?UT...

a message causing Indy mesasage decoder to enter infinite loop [Edit] #2
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. I have Indy 5231 which is not the latest but it is possibly it is a problem not addressed yet. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF...

a message causing Indy mesasage decoder to enter infinite loop [Edit] #2
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. I have Indy 5231 which is not the latest but it is possibly it is a problem not addressed yet. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF...

a message causing Indy mesasage decoder to enter infinite loop [Edit] #2
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. I have Indy 5231 which is not the latest but it is possibly it is a problem not addressed yet. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF...

a message causing Indy mesasage decoder to enter infinite loop [Edit] #2
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. I have Indy 5231 which is not the latest but it is possibly it is a problem not addressed yet. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF...

a message causing Indy mesasage decoder to enter infinite loop [Edit] #2
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. I have Indy 5231 which is not the latest but it is possibly it is a problem not addressed yet. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF...

a message causing Indy mesasage decoder to enter infinite loop [Edit] #2
I have a message which is obviously malformed and cannot be decoded properly (SUBJECT, FROM and TO are even decoded OK in Outlook), but in Indy it causes infinite loop. I'm OK with message not being decoded well due to being malformed (even though, subject and from line could probably be decoded good enough. But it should not infinite loop the decoder. I have Indy 5231 which is not the latest but it is possibly it is a problem not addressed yet. Here goes: {code} Subject: abc, =?UTF-8?B?TmVl?= =?UTF-8?B?ZCBDYQ==?= =?UTF-8?B?c2ggUXU=?= =?UTF-8?B?aWNrIEdl?= =?UTF-8?B?dCB1cHRvIFVT?= =?UTF...

Web resources about - A malformed message causing issue with the decoder [Edit] - embarcadero.cppbuilder.socket

Apple’s iCloud outage finally ends but some of those users left with malformed emails
The iCloud outage which began Tuesday that affected as many as 2% of Apple’s 200 Million iCloud users now appears to be over . However, some ...

Troubleshooting haproxy 502 errors related to malformed/large HTTP headers
... "GET /someurl HTTP/1.1" Another Google search revealed that PH means that haproxy rejected the header from the backend because it was malformed. ...

When Bad Heads Go Viral: How A Malformed Headline Is Skewing Our View Of The Patient's Role In Medicine ...
In the business of medicine, one of the brightest hopes is the potential for re-optimizing our spend around what patients want. That's important ...

415-million-year-old malformed fossil plankton reveal that heavy metal pollution might have contributed ...
Several Palaeozoic mass extinction events during the Ordovician and Silurian periods (ca. 485 to 420 to million years ago) shaped the evolution ...

Cisco Routing Systems Vulnerable to Malformed IPv6 Packet Attacks
Cisco Fixes DoS Vulnerability in IOS XR Software Cisco has released software updates to address a denial-of-service (DoS) vulnerability in IOS ...

Live Search's HTTP Headers Malformed
... shut down their tests that is causing fake referral data from being spewed out and now I am hearing reports that their HTTP headers are malformed. ...

Adam Voorhes photographs a rare collection of brains in his book, Malformed: Forgotten Brains of the ...
During previous editorial assignments at the University of Texas, Adam Voorhes had come across everything from a set of antique vacuum tubes ...

Fukushima's First Mutants Are Malformed Butterflies
The first creatures definitively shown to have mutated following the Fukushima meltdown are pale blue grass butterflies, but it's still a long ...

'AwSnap' malformed HTML exploit crashes Chrome in one click
... now, at least A bug in the most recent version of the Chrome allows miscreants to crash browser tabs simply by embedding a link with a malformed ...

Watch ‘Real Artists,’ a Semi-Animated Web Series Basically About Malformed Hipsters
We'll just let the YouTube description do our job for this one: "This first episode of Real Artists finds a number of characters engaged in their ...

Resources last updated: 12/12/2015 8:54:52 AM