"use encoding 'utf8'" breaks Net::LDAP

Graham Barr wrote:

> I find this very odd. The encoding pragma is not supposed to change how
> perl treats anything other than the sctual script in the file that the
> pragma is in. It allows you to write in any encoding and perl will
> translate that to utf8 when it parses the script.
>
> Maybe the pragma is doingmore than it should

Oh no!

"encoding" pragma does much more than just indicating the encoding of a
script. Here is the quote:

The encoding pragma also modifies the filehandle layers of STDIN and
STDOUT to the specified encoding.

By default, if strings operating under byte semantics and strings with
Unicode character data are concatenated, the new string will be created
by decoding the byte strings as ISO 8859-1 (Latin-1).
The encoding pragma changes this to use the specified encoding instead.

For example:

    use encoding 'utf8';
    my $string =3D chr(20000); # a Unicode string
    utf8::encode($string);   # now it's a UTF-8 encoded byte string
    # concatenate with another Unicode string
    print length($string . chr(20000));

Will print 2, because $string is upgraded as UTF-8. Without "use
encoding 'utf8';", it will print 4 instead, since $string is three
octets when interpreted as Latin-1.

--
Eugene Gladchenko,
EVG15-RIPE
0
gladchenko
9/23/2005 8:29:18 PM
perl.ldap 1268 articles. 0 followers. Follow

3 Replies
714 Views

Similar Articles

[PageSpeed] 19

On Sep 23, 2005, at 15:29 PM, Eugene Gladchenko wrote:
>> Maybe the pragma is doingmore than it should
>>
>
> Oh no!
>
> "encoding" pragma does much more than just indicating the encoding  
> of a
> script. Here is the quote:
>
> The encoding pragma also modifies the filehandle layers of STDIN and
> STDOUT to the specified encoding.

But we do not use STDIN/STDOUT so that should be a non-issue.

> By default, if strings operating under byte semantics and strings with
> Unicode character data are concatenated, the new string will be  
> created
> by decoding the byte strings as ISO 8859-1 (Latin-1).
> The encoding pragma changes this to use the specified encoding  
> instead.
>
> For example:
>
>     use encoding 'utf8';
>     my $string = chr(20000); # a Unicode string
>     utf8::encode($string);   # now it's a UTF-8 encoded byte string
>     # concatenate with another Unicode string
>     print length($string . chr(20000));
>
> Will print 2, because $string is upgraded as UTF-8. Without "use
> encoding 'utf8';", it will print 4 instead, since $string is three
> octets when interpreted as Latin-1.

Right, but the effect should be lexical. So other than strings that  
are upgraded as utf-8 being passed through to Net::LDAP as arguments,  
it should have no effect. And Net::LDAP should already handle UTF-8  
strings ok. But maybe there is one place that has been missed.

What I don't get is that the issue arises as a decode issue and all  
strings that are fed into the decode come from the socket and should  
not have this issue.

Graham.

0
gbarr
9/24/2005 2:28:19 PM
--Apple-Mail-3--1030054781
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

On Sep 24, 2005, at 09:28 AM, Graham Barr wrote:
> On Sep 23, 2005, at 15:29 PM, Eugene Gladchenko wrote:
>> Oh no!
>>
>> "encoding" pragma does much more than just indicating the encoding  
>> of a
>> script. Here is the quote:
>>
>> The encoding pragma also modifies the filehandle layers of STDIN and
>> STDOUT to the specified encoding.

The bug was in Convert::ASN1. Try the attached patch

Although I really think the bug was is in encoding. use encoding is  
changing how length etc is working in Convert::ASN1 and that is  
action at a distance

Graham.


--Apple-Mail-3--1030054781
Content-Transfer-Encoding: 7bit
Content-Type: application/octet-stream;
	x-unix-mode=0644;
	name="asn1.patch"
Content-Disposition: attachment;
	filename=asn1.patch

Index: Convert/ASN1/_decode.pm
===================================================================
--- Convert/ASN1/_decode.pm	(revision 96)
+++ Convert/ASN1/_decode.pm	(working copy)
@@ -5,10 +5,8 @@
 package Convert::ASN1;
 
 BEGIN {
-  unless (CHECK_UTF8) {
-    local $SIG{__DIE__};
-    eval { require bytes } and 'bytes'->import
-  }
+  local $SIG{__DIE__};
+  eval { require bytes and 'bytes'->import };
 }
 
 # These are the subs that do the decode, they are called with

--Apple-Mail-3--1030054781--
0
gbarr
9/24/2005 10:30:00 PM
On 9/25/05, Graham Barr <gbarr@pobox.com> wrote:

> The bug was in Convert::ASN1. Try the attached patch

Yes, it helped. Thank you!

--
Eugene Gladchenko
EVG15-RIPE
0
gladchenko
9/27/2005 3:14:14 PM
Reply:

Similar Artilces:

[Fwd: "use encoding 'utf8'" breaks Net::LDAP::RootDSE]
Hi, I hard a hard time tracking this error down, but it seems the two lines in my code broke my LDAP CGI script: use utf8; use encoding 'utf8'; Example to demonstrate: # perl -wd -e 'use encoding "utf8";use Net::LDAP; use Net::LDAP::RootDSE' Loading DB routines from perl5db.pl version 1.27 Editor support available. Enter h or `h h' for help, or `man perldebug' for more help. Debugged program terminated. Use q to quit or R to restart, use O inhibit_exit to avoid stopping after program termination, h q, h R or h O to get additiona...

Fwd: "use encoding 'utf8'" breaks Net::LDAP::RootDSE
Alexandre Jousset wrote: > I wrote a small test app like this: > ------------------------------------------------------- > #!/usr/bin/perl -w > > use strict; > use utf8; > use encoding 'utf8'; > > use Net::LDAP; > > my $ldap = Net::LDAP->new('localhost', version => 3); > > unless ($ldap) { > die "Could not connect to localhost: $@"; > } > > my ($dn, $passwd) = ('cn=admin,dc=exemple,dc=com', 'mypass'); > > my $mesg = $ldap->bind(dn => $dn, password =&g...

[ID 20020307.006] [BUG][use encoding] use encoding 'greek'; print "not" unless pack("U*", 0x3af) eq pack("C*", 0xdf) prints "not"!
This is a bug report for perl from "Anton Tagunov" <tagunov@motor.ru>, generated with the help of perlbug 1.33 running under perl v5.7.3. ----------------------------------------------------------------- Hello, developers! Hello, Jarkko! Please consider the following test being added: --- lib/encoding.t.orig Fri Feb 8 07:12:05 2002 +++ lib/encoding.t Thu Mar 7 19:38:39 2002 @@ -1,4 +1,4 @@ -print "1..19\n"; +print "1..20\n"; BEGIN { if (ord("A") == 193) { @@ -89,3 +89,7 @@ print "not " unless "\x...

RFC: interactions between "use bytes", "use locale", and "use feature 'unicode_strings'
I am waiting for blead to reopen before I submit a patch for extending feature unicode_strings to matching \s, and \w. In documenting it, it occurred to me that the existing implementation is wrong, along with the existing interaction between use bytes and use locale. To refresh your memory, "use feature 'unicode_strings'" is supposed to mean that even non-utf8 data is to be considered to have Unicode semantics. In practice, this only affects the characters from 128-255. It is new in 5.12, and is implemented there only on functions that change case, such a...

I'm getting an error using NavigateURL ='<%# Eval("Url") %>', it works when using NavigateURL="intro.aspx"
I'm getting an error using NavigateURL ='<%# Eval("Url") %>', it works when I use NavigateURL="intro.aspx" Server Error in '/orthoassist' Application. The resource cannot be found. Description: HTTP 404. The resource you are looking for (or one of its dependencies) could have been removed, had its name changed, or is temporarily unavailable.  Please review the following URL and make sure that it is spelled correctly. Requested URL: /orthoassist/intro.aspx <asp:HyperLink runat="server" ID="hl1" Text=...

Can't locate object method "disconnect" via package "Net::LDAP"
------_=_NextPart_001_01C85279.EDDDD442 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable Hello, I am trying to disconnect the ldap connection handle in a perl program. Here is the sequence that I am following: connect and bind unbind $ldap->disconnect(); I get the error Can't locate object method "disconnect" via package "Net::LDAP" How should I use disconnect?=20 What is the difference between an unbind and disconnect? Unbind seems to work just fine. I am just curious. Thanks, Praveen ...

RFC: Make "use 5.14" select "use re '/a' " ?
I have pushed to blead (with Jesse's permission) the last commits (besides doc and any bug fixes) that enable the /a regular expression modifier. Presuming it works ok, should this become the default method of operation, signified by a "use 5.014" ? On 1/17/11 11:02 AM, Karl Williamson wrote: > Presuming it works ok, should this become the default method of > operation, signified by a "use 5.014" ? So when I upgrade my code, I have to put "no re '/a'" in every file? Why, oh why? On Mon, 17 Jan 2011, Elliot Shank wrote: > ...

WARNING: "Application 'sybinit' is using character set 'iso_1', but the Replication Server is using character set 'utf8'
RepServer 12.6 ESD#2 RSSD on ASE 12.5.1 ESD#2 I have the following when I created my first RepServer: 2004/06/22 11:22:04 waiting for server 'SYDNEY_RS' to boot... 2004/06/22 11:22:09 Task succeeded: start the Replication Server. 2004/06/22 11:22:09 Running task: configure the Replication Server. 2004/06/22 11:22:09 SQL Server message: msg 14125, level 11, state 0 2004/06/22 11:24:18 WARNING: "Application 'sybinit' is using character set 'iso_1', but the Replication Server is using character set 'utf8&#...

" Cannot find the script or external object that implements prefix 'fn' " error when I use "fn" prefix in xslt and I call it with ASP.NET.2.0
Hi, I’m trying to display the contents of an xml file using an xslt file. I’m using ASP.NET.2.0 and the xsltCompiledTransform method in order to achieve the transformation. Everything works fine when I don’t have xPath functions in my xslt but when I declare the namespace ‘xmlns:fn=http://www.w3.org/2005/02/xpath-functions' and I use Xpath functions in my xslt file, it comes up with the following error message : ‘Cannot find the script or external object that implements prefix 'fn'. Just to mention I use VB as a scripting language in my ASP page. Could anyone please tell me w...

Replace(txt,"'","''") in code-behind
i'm creating a file named client.vb using my wordpad. I'm using this as a code-behind sort of thing. The problem is I want to remove the ' from my strings so that i can put them in SQL statements. The Replace doesn't work, is there any other way to this. thanks. Can you show me the code sample you have for the string.Replace() method? That should work for you. You should instead use Sql Parameters to compile and send your query to the db. One of the many benefits of doing so is that you don't need to worry about the single quotes.-- daver True. However, I'd like to know why you are ha...

'Entropy', a "net inside the net" ?
Just tumbled upon this, maybe old news. Comments welcome (f/u grc.privacy) : Entropy is a software, which creates a "net inside the net" (inside the internet) for its users. This software can be used both by *nix- and Windows-users. Entropy is "Free Software" (under GPL) and can be downloaded here: http://entropy.stop1984.com/ -- Ninho On Wed, 12 Sep 2007 12:03:30 +0200, in grc.linkfarm,grc.privacy, Ninho wrote: > Just tumbled upon this, maybe old news. Comments welcome (f/u grc.privacy) : > > Entropy is a software, which creates a "ne...

"The name 'sizes' does not exist in the namespace 'ASP.NET.StarterKit.Commerce.ShoppingCart'"
I am having the following error when I build: "The name 'sizes' does not exist in the namespace 'ASP.NET.StarterKit.Commerce.ShoppingCart'" Not sure what I need to do to fix it. I added a collum to the shopping cart and now the Update and Remove do not work. I have made the adjustments in my Stored Procedures and in the ShoppingCartDB, both in the UpdateItem and RemoveItem areas. Not sure how to fix this one. Please help. Here is the code behind for the shopping cart page. namespace ASPNET.StarterKit.Commerce { public class ShoppingCart : System.Web.UI.Page { protec...

Using cdate function but got error "Conversion from string " " to type 'Date' is not valid."
I want to control user's input in Due date text box like below to make sure that user does not input date earlier than today: If Me.txtDueDate.Text <> "" Then      If (DateDiff("d", CDate(Me.txtDueDate.Text), (Date.Today)) > 0) Then     // Message: due day should not be earlier than today      exit sub End If  The problem is: if user did not input anything in txtDueDate box, I still got error: Conversion from string "  " to type 'Date' is not valid How to fix it? Help!!!   I'd use a RequireFie...

What does "use vars '@ISA';@ISA = 'LWP::UserAgent';my $agent = __PACKAGE__->new;"
------=_NextPart_000_0009_01C54F1B.7B7BCF30 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Can someone help me understand Lincoln Stein's code I found in his book on network programming with perl? This is the beginning of a screen scraper. I thought ISA was an array of ancestors used exclusively for defining descendant classes. I guess I'm wrong. What the heck is going on here? I understand the "use" statements, but the subsequent statements baffle me. Thanks, Siegfried use LWP; use PromptU...

Web resources about - "use encoding 'utf8'" breaks Net::LDAP - perl.ldap

Resources last updated: 1/23/2016 7:46:58 PM